Jul 06 22:24:26.272662 ip-10-0-132-23 systemd[1]: Starting Kubernetes Kubelet... Jul 06 22:24:26.649218 ip-10-0-132-23 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:24:26.649218 ip-10-0-132-23 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 22:24:26.649218 ip-10-0-132-23 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:24:26.649218 ip-10-0-132-23 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 22:24:26.649218 ip-10-0-132-23 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:24:26.650217 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.650140 2572 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 22:24:26.654891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654877 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:24:26.654891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654892 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654896 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654899 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654902 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654904 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654907 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654909 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654912 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654914 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654917 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654919 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654922 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654924 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654927 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654929 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654941 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654943 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654946 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654949 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654951 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:24:26.654952 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654954 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654957 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654959 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654962 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654965 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654968 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654970 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654973 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654975 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654978 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654981 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654983 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654986 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654988 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654991 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654993 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654996 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.654998 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655001 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655004 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:24:26.655472 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655006 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655009 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655011 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655014 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655016 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655018 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655021 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655023 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655025 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655028 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655030 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655032 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655035 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655037 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655041 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655043 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655045 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655050 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655054 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655057 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:24:26.655953 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655060 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655063 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655066 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655069 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655071 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655074 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655077 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655080 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655082 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655084 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655088 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655090 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655093 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655095 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655098 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655102 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655105 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655108 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655110 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:24:26.656469 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655113 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655116 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655119 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655122 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655125 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655129 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655134 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655137 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655140 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655143 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655145 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655148 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655151 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655153 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655155 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655158 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655161 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655164 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655166 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655867 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:24:26.656961 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655874 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655877 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655879 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655882 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655885 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655888 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655890 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655893 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655895 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655898 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655901 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655903 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655906 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655908 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655911 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655914 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655916 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655919 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655922 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655925 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:24:26.657455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655927 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655930 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655932 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655935 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655938 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655940 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655943 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655945 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655948 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655950 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655953 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655956 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655959 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655961 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655964 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655967 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655970 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655973 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655977 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:24:26.657975 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655981 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655983 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655986 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655989 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655991 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655994 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655996 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.655999 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656001 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656004 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656007 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656010 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656013 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656016 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656018 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656021 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656024 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656027 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656029 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656032 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:24:26.658451 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656034 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656037 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656040 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656042 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656044 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656047 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656050 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656052 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656055 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656057 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656060 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656063 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656065 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656068 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656071 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656074 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656077 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656080 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656083 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:24:26.658929 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656086 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656090 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656092 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656095 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656098 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656101 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656104 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656107 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656109 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656112 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656115 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656117 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656121 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656125 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656127 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656130 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656132 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656135 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656137 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656140 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:24:26.659390 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656519 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656525 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656528 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656531 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656534 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656537 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656543 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656545 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656548 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656551 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656553 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656556 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656559 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656561 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656564 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656566 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656570 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656573 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656575 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656578 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:24:26.659891 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656580 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656583 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656586 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656588 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656592 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656595 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656598 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656601 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656604 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656606 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656609 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656611 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656614 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656616 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656618 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656621 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656623 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656626 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656630 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656633 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:24:26.660477 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656636 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656638 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656641 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656643 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656646 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656648 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656651 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656653 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656656 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656659 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656663 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656666 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656669 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656672 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656675 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656678 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656680 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656683 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656686 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:24:26.660972 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656689 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656692 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656694 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656697 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656699 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656702 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656705 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656707 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656709 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656712 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656714 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656719 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656723 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656725 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656727 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656730 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656733 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656735 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656738 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656740 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:24:26.661444 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656742 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656746 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656749 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656752 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656756 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656759 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656761 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656764 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656767 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656770 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656772 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656775 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656777 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656780 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656782 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656785 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656787 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656790 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656792 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:24:26.661958 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.656794 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656860 2572 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656866 2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656872 2572 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656877 2572 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656882 2572 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656885 2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656889 2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656893 2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656896 2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656899 2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656902 2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656905 2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656908 2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656912 2572 flags.go:64] FLAG: --cgroup-root="" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656915 2572 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656918 2572 flags.go:64] FLAG: --client-ca-file="" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656921 2572 flags.go:64] FLAG: --cloud-provider="external" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656923 2572 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656928 2572 flags.go:64] FLAG: --cluster-domain="" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656931 2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656934 2572 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656937 2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656941 2572 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 22:24:26.662416 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656944 2572 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656947 2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656950 2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656953 2572 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656956 2572 flags.go:64] FLAG: --contention-profiling="false" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656959 2572 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656962 2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656965 2572 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656967 2572 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656971 2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656974 2572 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656977 2572 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656981 2572 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656984 2572 flags.go:64] FLAG: --enable-server="true" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656987 2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656992 2572 flags.go:64] FLAG: --event-burst="100" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656995 2572 flags.go:64] FLAG: --event-qps="50" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.656997 2572 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657000 2572 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657003 2572 flags.go:64] FLAG: --eviction-hard="" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657007 2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657009 2572 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657012 2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657015 2572 flags.go:64] FLAG: --eviction-soft="" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657018 2572 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 22:24:26.663016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657021 2572 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657024 2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657026 2572 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657029 2572 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657032 2572 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657034 2572 flags.go:64] FLAG: --feature-gates="" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657038 2572 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657040 2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657043 2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657046 2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657049 2572 flags.go:64] FLAG: --healthz-port="10248" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657052 2572 flags.go:64] FLAG: --help="false" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657054 2572 flags.go:64] FLAG: --hostname-override="ip-10-0-132-23.ec2.internal" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657057 2572 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657060 2572 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657062 2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657065 2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657069 2572 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657071 2572 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657075 2572 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657078 2572 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657081 2572 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657084 2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657087 2572 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 22:24:26.663631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657090 2572 flags.go:64] FLAG: --kube-reserved="" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657093 2572 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657095 2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657098 2572 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657101 2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657104 2572 flags.go:64] FLAG: --lock-file="" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657106 2572 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657109 2572 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657112 2572 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657117 2572 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657120 2572 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657122 2572 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657125 2572 flags.go:64] FLAG: --logging-format="text" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657128 2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657131 2572 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657134 2572 flags.go:64] FLAG: --manifest-url="" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657136 2572 flags.go:64] FLAG: --manifest-url-header="" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657141 2572 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657144 2572 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657147 2572 flags.go:64] FLAG: --max-pods="110" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657150 2572 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657153 2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657155 2572 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657158 2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657161 2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 22:24:26.664222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657163 2572 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657166 2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657174 2572 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657177 2572 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657180 2572 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657183 2572 flags.go:64] FLAG: --pod-cidr="" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657186 2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657191 2572 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657194 2572 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657197 2572 flags.go:64] FLAG: --pods-per-core="0" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657200 2572 flags.go:64] FLAG: --port="10250" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657203 2572 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657206 2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b4d36fc7b77abd50" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657210 2572 flags.go:64] FLAG: --qos-reserved="" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657212 2572 flags.go:64] FLAG: --read-only-port="10255" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657215 2572 flags.go:64] FLAG: --register-node="true" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657218 2572 flags.go:64] FLAG: --register-with-taints="" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657221 2572 flags.go:64] FLAG: --registry-burst="10" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657224 2572 flags.go:64] FLAG: --registry-qps="5" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657227 2572 flags.go:64] FLAG: --reserved-cpus="" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657229 2572 flags.go:64] FLAG: --reserved-memory="" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657233 2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657235 2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657238 2572 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657241 2572 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 22:24:26.664869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657244 2572 flags.go:64] FLAG: --runonce="false" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657246 2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657249 2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657252 2572 flags.go:64] FLAG: --seccomp-default="false" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657255 2572 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657257 2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657260 2572 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657263 2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657266 2572 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657269 2572 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657272 2572 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657275 2572 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657278 2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657281 2572 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657284 2572 flags.go:64] FLAG: --system-cgroups="" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657287 2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657292 2572 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657294 2572 flags.go:64] FLAG: --tls-cert-file="" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657297 2572 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657302 2572 flags.go:64] FLAG: --tls-min-version="" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657305 2572 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657307 2572 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657310 2572 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657313 2572 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657316 2572 flags.go:64] FLAG: --v="2" Jul 06 22:24:26.665475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657319 2572 flags.go:64] FLAG: --version="false" Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657323 2572 flags.go:64] FLAG: --vmodule="" Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657327 2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.657330 2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657453 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657457 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657460 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657463 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657466 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657468 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657471 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657474 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657476 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657478 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657481 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657483 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657486 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657490 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657492 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657495 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:24:26.666056 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657497 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657500 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657503 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657505 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657508 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657512 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657515 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657517 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657520 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657522 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657525 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657527 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657530 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657532 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657536 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657539 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657542 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657544 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657547 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:24:26.666584 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657551 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657554 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657557 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657560 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657563 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657565 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657568 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657570 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657573 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657575 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657579 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657581 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657584 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657587 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657589 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657592 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657594 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657597 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657601 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657604 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:24:26.667053 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657606 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657608 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657611 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657613 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657616 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657618 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657621 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657623 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657626 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657628 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657631 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657633 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657636 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657638 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657641 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657643 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657646 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657648 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657650 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657653 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:24:26.667563 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657655 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657658 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657661 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657664 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657666 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657669 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657671 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657674 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657677 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657680 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657683 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657686 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657688 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657690 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657693 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657695 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657698 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657700 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657702 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657706 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:24:26.668042 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657708 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657711 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657714 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.657717 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.658290 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.665445 2572 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.665464 2572 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.665482 2572 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.665488 2572 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665567 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665573 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665576 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665579 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665581 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:24:26.668597 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665584 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665586 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665591 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665595 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665598 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665600 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665602 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665605 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665608 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665610 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665613 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665615 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665618 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665621 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665623 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665626 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665628 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665631 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665633 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:24:26.668942 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665636 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665638 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665641 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665643 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665646 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665650 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665653 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665656 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665658 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665661 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665663 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665666 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665668 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665671 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665673 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665676 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665678 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665681 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665683 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665686 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:24:26.669391 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665688 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665691 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665693 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665696 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665698 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665701 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665704 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665706 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665709 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665711 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665714 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665716 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665719 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665721 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665724 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665726 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665729 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665733 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665736 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:24:26.669860 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665740 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665744 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665747 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665750 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665752 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665755 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665757 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665760 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665762 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665765 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665767 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665770 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665773 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665776 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665778 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665781 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665783 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665786 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665788 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665791 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:24:26.670310 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665793 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665796 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665798 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665800 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665803 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665805 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665808 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665810 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665813 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665815 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665818 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665821 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665824 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665827 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665829 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:24:26.670793 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665833 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.665839 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665968 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665973 2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665977 2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665980 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665982 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665985 2572 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665988 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665991 2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665994 2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665996 2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.665999 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666001 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666004 2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:24:26.671159 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666006 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666009 2572 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666011 2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666014 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666016 2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666019 2572 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666021 2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666024 2572 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666026 2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666029 2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666031 2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666034 2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666036 2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666039 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666042 2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666045 2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666048 2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666051 2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666053 2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666056 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:24:26.671546 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666058 2572 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666061 2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666064 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666066 2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666069 2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666072 2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666074 2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666077 2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666079 2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666082 2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666085 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666087 2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666090 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666092 2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666095 2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666097 2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666100 2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666102 2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666105 2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666107 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:24:26.672009 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666110 2572 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666112 2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666116 2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666119 2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666122 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666125 2572 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666128 2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666131 2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666134 2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666137 2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666139 2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666141 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666144 2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666146 2572 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666149 2572 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666151 2572 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666154 2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666158 2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666160 2572 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:24:26.672550 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666163 2572 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666165 2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666168 2572 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666170 2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666172 2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666175 2572 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666177 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666180 2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666183 2572 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666186 2572 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666189 2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666192 2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666194 2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666197 2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666199 2572 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666202 2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666205 2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666207 2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666210 2572 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:24:26.672986 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666212 2572 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666215 2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666217 2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666220 2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666223 2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666225 2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666227 2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:26.666230 2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.666235 2572 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.666346 2572 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.670193 2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.672037 2572 server.go:1019] "Starting client certificate rotation" Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.672139 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:24:26.673442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.672175 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:24:26.676673 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.676314 2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:24:26.678212 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.678193 2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:24:26.686670 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.686655 2572 log.go:25] "Validated CRI v1 runtime API" Jul 06 22:24:26.689903 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.689886 2572 log.go:25] "Validated CRI v1 image API" Jul 06 22:24:26.690689 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.690672 2572 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 22:24:26.694579 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.694559 2572 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 c4933467-cd1d-4524-bbb8-ee3360900423:/dev/nvme0n1p3 c986dd12-d483-4769-b235-83b98be6f9fa:/dev/nvme0n1p4] Jul 06 22:24:26.694649 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.694579 2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 22:24:26.700142 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.699977 2572 manager.go:216] Machine: {Timestamp:2026-07-06 22:24:26.698817784 +0000 UTC m=+0.311847741 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103320 MemoryCapacity:32812183552 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e32b121cbd5046a876dfe0d9c277a SystemUUID:ec2e32b1-21cb-d504-6a87-6dfe0d9c277a BootID:ae3acd9d-9675-41f2-97e3-55bea44036b6 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406093824 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:27:91:15:08:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:27:91:15:08:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:6a:5f:79:1f:f9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812183552 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 22:24:26.700142 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.700137 2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 22:24:26.700251 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.700232 2572 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 22:24:26.703528 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.703502 2572 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 22:24:26.703653 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.703531 2572 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 22:24:26.704885 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.704873 2572 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 22:24:26.704922 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.704889 2572 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 22:24:26.704922 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.704897 2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:24:26.705543 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.705533 2572 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:24:26.705744 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.705730 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:24:26.708095 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.708082 2572 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 22:24:26.711525 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.711512 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:24:26.712279 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712269 2572 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 22:24:26.712311 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712299 2572 kubelet.go:486] "Attempting to sync node with API server" Jul 06 22:24:26.712311 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712309 2572 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 22:24:26.712360 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712323 2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 22:24:26.712360 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712331 2572 kubelet.go:398] "Adding apiserver pod source" Jul 06 22:24:26.712360 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712339 2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 22:24:26.712737 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712724 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:24:26.712774 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.712744 2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:24:26.714756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.714729 2572 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 22:24:26.718483 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.718446 2572 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 22:24:26.718577 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.718512 2572 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 22:24:26.720748 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720734 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720752 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720758 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720763 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720773 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720778 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720786 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720791 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720799 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720805 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720813 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 22:24:26.720818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.720818 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 22:24:26.721488 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.721474 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 22:24:26.721488 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.721486 2572 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 22:24:26.724479 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.724467 2572 server.go:1268] "Started kubelet" Jul 06 22:24:26.724589 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.724538 2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 22:24:26.724589 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.724536 2572 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 22:24:26.724734 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.724599 2572 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 22:24:26.724980 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.724953 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 22:24:26.724980 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.724967 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 22:24:26.724980 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.724976 2572 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 22:24:26.725343 ip-10-0-132-23 systemd[1]: Started Kubernetes Kubelet. Jul 06 22:24:26.726347 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.726329 2572 server.go:310] "Adding debug handlers to kubelet server" Jul 06 22:24:26.726584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.726569 2572 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 22:24:26.730326 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.730308 2572 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 22:24:26.735348 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.734401 2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-23.ec2.internal.18bfd3a130d49f52 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-23.ec2.internal,UID:ip-10-0-132-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-23.ec2.internal,},FirstTimestamp:2026-07-06 22:24:26.724441938 +0000 UTC m=+0.337471887,LastTimestamp:2026-07-06 22:24:26.724441938 +0000 UTC m=+0.337471887,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-23.ec2.internal,}" Jul 06 22:24:26.740977 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.740962 2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 22:24:26.741066 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.741019 2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 22:24:26.741125 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.741067 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:26.741927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.741684 2572 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 22:24:26.741927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.741685 2572 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 22:24:26.741927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.741762 2572 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 22:24:26.741927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.741851 2572 reconstruct.go:97] "Volume reconstruction finished" Jul 06 22:24:26.741927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.741860 2572 reconciler.go:29] "Reconciler: start to sync state" Jul 06 22:24:26.742906 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742631 2572 factory.go:153] Registering CRI-O factory Jul 06 22:24:26.742906 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742812 2572 factory.go:223] Registration of the crio container factory successfully Jul 06 22:24:26.742906 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742894 2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 22:24:26.742906 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742902 2572 factory.go:55] Registering systemd factory Jul 06 22:24:26.743100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742930 2572 factory.go:223] Registration of the systemd container factory successfully Jul 06 22:24:26.743100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742951 2572 factory.go:103] Registering Raw factory Jul 06 22:24:26.743100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.742964 2572 manager.go:1196] Started watching for new ooms in manager Jul 06 22:24:26.743627 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.743590 2572 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 22:24:26.743860 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.743827 2572 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 22:24:26.744750 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.744734 2572 manager.go:349] Starting recovery of all containers Jul 06 22:24:26.746279 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.746248 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cfj6m" Jul 06 22:24:26.754850 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.754830 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cfj6m" Jul 06 22:24:26.755148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.755136 2572 manager.go:354] Recovery completed Jul 06 22:24:26.758997 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.758983 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:26.761030 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761017 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:26.761102 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761049 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:26.761102 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761061 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:26.761543 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761529 2572 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 22:24:26.761543 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761542 2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 22:24:26.761641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761559 2572 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:24:26.761641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.761532 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:26.762256 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.762242 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:26.762325 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.762266 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:26.762325 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.762277 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:26.762559 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.762549 2572 policy_none.go:49] "None policy: Start" Jul 06 22:24:26.762604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.762563 2572 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 22:24:26.762604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.762574 2572 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 22:24:26.763501 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.763483 2572 policy_none.go:47] "Start" Jul 06 22:24:26.811931 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.811912 2572 manager.go:337] "Starting Device Plugin manager" Jul 06 22:24:26.812007 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.811944 2572 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 22:24:26.812007 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.811958 2572 server.go:87] "Starting device plugin registration server" Jul 06 22:24:26.812205 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.812183 2572 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 22:24:26.812205 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.812196 2572 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 22:24:26.812317 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.812300 2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 22:24:26.812439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.812422 2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 22:24:26.812439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.812437 2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 22:24:26.813622 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.813603 2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 22:24:26.813686 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.813648 2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:26.912660 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.912593 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:26.913858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.913682 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:26.913858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.913710 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:26.913858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.913720 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:26.913858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.913746 2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-23.ec2.internal" Jul 06 22:24:26.919480 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.919465 2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-23.ec2.internal" Jul 06 22:24:26.919547 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.919495 2572 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-23.ec2.internal\": node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:26.931281 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.931254 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 22:24:26.932606 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.932592 2572 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 22:24:26.932698 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.932624 2572 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 22:24:26.932698 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.932643 2572 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 22:24:26.932698 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.932687 2572 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 22:24:26.934768 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:26.934753 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:26.935968 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:26.935953 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:27.033701 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.033660 2572 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal"] Jul 06 22:24:27.033820 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.033758 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:27.035338 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.035323 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:27.035443 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.035352 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:27.035443 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.035365 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:27.036426 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:27.036410 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:27.036791 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.036780 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:27.036902 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.036889 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.036962 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.036938 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:27.037629 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.037606 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:27.037629 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.037625 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:27.037763 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.037641 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:27.037763 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.037646 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:27.037763 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.037658 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:27.037763 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.037662 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:27.038852 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.038834 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.038919 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.038861 2572 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:24:27.039602 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.039587 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:24:27.039651 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.039614 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:24:27.039688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.039658 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:24:27.043493 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.043305 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d8e8123e63af15df39698cf832a3600-config\") pod \"kube-apiserver-proxy-ip-10-0-132-23.ec2.internal\" (UID: \"4d8e8123e63af15df39698cf832a3600\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.043493 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.043341 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f731d16102760bfb05c5d6c7046a2b9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"f731d16102760bfb05c5d6c7046a2b9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.043493 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.043388 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f731d16102760bfb05c5d6c7046a2b9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"f731d16102760bfb05c5d6c7046a2b9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.062911 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:27.062887 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-23.ec2.internal\" not found" node="ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.067303 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:27.067287 2572 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-23.ec2.internal\" not found" node="ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.136989 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:27.136964 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:27.144317 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.144294 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f731d16102760bfb05c5d6c7046a2b9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"f731d16102760bfb05c5d6c7046a2b9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.144421 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.144344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f731d16102760bfb05c5d6c7046a2b9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"f731d16102760bfb05c5d6c7046a2b9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.144421 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.144392 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d8e8123e63af15df39698cf832a3600-config\") pod \"kube-apiserver-proxy-ip-10-0-132-23.ec2.internal\" (UID: \"4d8e8123e63af15df39698cf832a3600\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.144421 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.144402 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f731d16102760bfb05c5d6c7046a2b9d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"f731d16102760bfb05c5d6c7046a2b9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.144550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.144454 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f731d16102760bfb05c5d6c7046a2b9d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal\" (UID: \"f731d16102760bfb05c5d6c7046a2b9d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.144550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.144469 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4d8e8123e63af15df39698cf832a3600-config\") pod \"kube-apiserver-proxy-ip-10-0-132-23.ec2.internal\" (UID: \"4d8e8123e63af15df39698cf832a3600\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.237771 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:27.237703 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:27.338241 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:27.338211 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:27.365740 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.365721 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:27.371148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:27.371131 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" Jul 06 22:24:28.747672 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:28.747652 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:28.848307 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:28.848183 2572 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-23.ec2.internal\" not found" Jul 06 22:24:28.886789 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.886764 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:28.906646 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.906629 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:28.950039 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.950018 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" Jul 06 22:24:28.959200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.959185 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:24:28.961176 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.961155 2572 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" Jul 06 22:24:28.971838 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.971822 2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:24:28.979975 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.979963 2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 22:24:28.980070 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.980055 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:24:28.980110 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.980090 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:24:28.980155 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:28.980110 2572 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:24:29.022345 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.022305 2572 apiserver.go:52] "Watching apiserver" Jul 06 22:24:29.041589 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.041556 2572 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 22:24:29.042691 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.042669 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-n88h9","openshift-image-registry/node-ca-ddk2j","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal","openshift-multus/multus-additional-cni-plugins-lkm55","openshift-multus/multus-km9xf","kube-system/konnectivity-agent-krqkm","kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal","openshift-multus/network-metrics-daemon-2sj22","openshift-network-diagnostics/network-check-target-xrmgf","openshift-network-operator/iptables-alerter-q5whp","openshift-ovn-kubernetes/ovnkube-node-7tdp7","kube-system/global-pull-secret-syncer-vlvg2","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr","openshift-cluster-node-tuning-operator/tuned-2cg9w"] Jul 06 22:24:29.044436 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.044420 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.045594 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.045578 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.047078 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.047062 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.047172 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.047155 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.047249 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.047222 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cbd6w\"" Jul 06 22:24:29.047578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.047555 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.048014 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.047995 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 22:24:29.048095 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.048068 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.048134 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.048124 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j7dvx\"" Jul 06 22:24:29.048268 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.048254 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.048687 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.048675 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.050028 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050008 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.050123 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050037 2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 22:24:29.050542 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050344 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m2klg\"" Jul 06 22:24:29.050542 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050409 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 22:24:29.050542 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050420 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 22:24:29.050542 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050426 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.050542 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050457 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 22:24:29.050865 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.050842 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.051208 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.051181 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 22:24:29.051315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.051218 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rhmpk\"" Jul 06 22:24:29.051965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.051950 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.052029 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.052002 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:29.052595 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.052580 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ghpb2\"" Jul 06 22:24:29.052750 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.052735 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 22:24:29.052996 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.052982 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 22:24:29.053279 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.053262 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:29.053345 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.053329 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:29.054604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.054578 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.056058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056037 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.056510 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056490 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4338e08-6387-4995-8195-d0ab26a18e5b-host\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.056586 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5w2x\" (UniqueName: \"kubernetes.io/projected/d4338e08-6387-4995-8195-d0ab26a18e5b-kube-api-access-j5w2x\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.056586 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056552 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-cni-binary-copy\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.056586 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056568 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-cni-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056586 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056584 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-socket-dir-parent\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056611 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.056700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056634 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-cni-binary-copy\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056649 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-netns\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056663 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-cni-bin\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056678 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94e520a8-48a2-4f5e-9dce-f6487ff55798-hosts-file\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056711 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jltx\" (UniqueName: \"kubernetes.io/projected/94e520a8-48a2-4f5e-9dce-f6487ff55798-kube-api-access-9jltx\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056733 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-kubelet\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056753 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-os-release\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056788 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5pqp\" (UniqueName: \"kubernetes.io/projected/53b05de2-d740-42a4-886a-f10ec45dbef5-kube-api-access-p5pqp\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056813 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-os-release\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056833 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-cni-multus\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.056853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-hostroot\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056862 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-conf-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056882 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-daemon-config\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056901 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056916 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056952 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-k8s-cni-cncf-io\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056974 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-multus-certs\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.056991 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-etc-kubernetes\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057008 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gxs2\" (UniqueName: \"kubernetes.io/projected/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-kube-api-access-8gxs2\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057023 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f6c5d5e-2d4b-4b0b-8eae-12039897236e-agent-certs\") pod \"konnectivity-agent-krqkm\" (UID: \"4f6c5d5e-2d4b-4b0b-8eae-12039897236e\") " pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4338e08-6387-4995-8195-d0ab26a18e5b-serviceca\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-system-cni-dir\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.057109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057112 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-system-cni-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057144 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f6c5d5e-2d4b-4b0b-8eae-12039897236e-konnectivity-ca\") pod \"konnectivity-agent-krqkm\" (UID: \"4f6c5d5e-2d4b-4b0b-8eae-12039897236e\") " pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057170 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94e520a8-48a2-4f5e-9dce-f6487ff55798-tmp-dir\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057195 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-cnibin\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057221 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057250 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-cnibin\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057228 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 22:24:29.057550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.057290 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh7wm\" (UniqueName: \"kubernetes.io/projected/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-kube-api-access-qh7wm\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.058346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.058052 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rbsjk\"" Jul 06 22:24:29.058346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.058076 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.058346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.058084 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.058346 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.058161 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:29.058910 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.058890 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 22:24:29.059102 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.059087 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.059189 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.059173 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 22:24:29.059340 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.059324 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-52nh2\"" Jul 06 22:24:29.059386 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.059356 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 22:24:29.059640 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.059623 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 22:24:29.059687 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.059663 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.060339 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.060318 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.061584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.061567 2572 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:24:29.061877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.061865 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.062978 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.062958 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.063055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.063016 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h7vc2\"" Jul 06 22:24:29.063183 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.063166 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 22:24:29.063776 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.063761 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.064648 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.064634 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 22:24:29.064731 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.064648 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kdvfk\"" Jul 06 22:24:29.064731 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.064680 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:24:29.069079 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.069044 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 22:19:28 +0000 UTC" deadline="2028-01-14 08:32:44.358066642 +0000 UTC" Jul 06 22:24:29.069143 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.069079 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13354h8m15.288990989s" Jul 06 22:24:29.083725 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.083700 2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ghsfg" Jul 06 22:24:29.092746 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.092727 2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ghsfg" Jul 06 22:24:29.151544 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.151525 2572 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157503 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5w2x\" (UniqueName: \"kubernetes.io/projected/d4338e08-6387-4995-8195-d0ab26a18e5b-kube-api-access-j5w2x\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157545 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysctl-d\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157570 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-sys\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157594 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-host\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5g8n\" (UniqueName: \"kubernetes.io/projected/3c5f9cb7-f0f2-470d-811a-6a0015455d42-kube-api-access-j5g8n\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157661 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-socket-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-cni-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157773 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-cni-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157776 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-socket-dir-parent\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157804 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-var-lib-kubelet\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157829 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157855 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-netns\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157878 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-cni-bin\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157905 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-run-netns\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157942 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-netns\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.158247 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.157990 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-cni-bin\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158031 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-systemd\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158060 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-env-overrides\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158087 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pf65\" (UniqueName: \"kubernetes.io/projected/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-kube-api-access-9pf65\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158152 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94e520a8-48a2-4f5e-9dce-f6487ff55798-hosts-file\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158190 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-kubelet\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158217 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-modprobe-d\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158268 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-lib-modules\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158289 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/94e520a8-48a2-4f5e-9dce-f6487ff55798-hosts-file\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158293 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-cni-netd\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158334 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-os-release\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158360 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-kubelet\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158397 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5pqp\" (UniqueName: \"kubernetes.io/projected/53b05de2-d740-42a4-886a-f10ec45dbef5-kube-api-access-p5pqp\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158424 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-hostroot\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158453 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-daemon-config\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158474 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158477 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysctl-conf\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.159081 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158513 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-hostroot\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158534 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-systemd-units\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158619 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-ovn\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158620 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-os-release\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158667 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-node-log\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158691 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-socket-dir-parent\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158695 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158722 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-k8s-cni-cncf-io\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158764 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-multus-certs\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158790 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gxs2\" (UniqueName: \"kubernetes.io/projected/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-kube-api-access-8gxs2\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158826 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158830 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-k8s-cni-cncf-io\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158852 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158879 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158897 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158906 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw6zw\" (UniqueName: \"kubernetes.io/projected/64fbf0a0-0f71-4641-ac1f-b88539056744-kube-api-access-xw6zw\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158941 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4338e08-6387-4995-8195-d0ab26a18e5b-serviceca\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.159933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158967 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-system-cni-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.158992 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f6c5d5e-2d4b-4b0b-8eae-12039897236e-konnectivity-ca\") pod \"konnectivity-agent-krqkm\" (UID: \"4f6c5d5e-2d4b-4b0b-8eae-12039897236e\") " pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.159008 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159024 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-cni-bin\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159057 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94e520a8-48a2-4f5e-9dce-f6487ff55798-tmp-dir\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.159087 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:29.659051342 +0000 UTC m=+1.963219985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159117 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-cnibin\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159124 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-system-cni-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159142 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysconfig\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159030 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-daemon-config\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159167 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982e3dc6-626c-4fd5-9616-6670f54a108c-tmp\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159083 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-run-multus-certs\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159210 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-registration-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159280 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4338e08-6387-4995-8195-d0ab26a18e5b-host\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159309 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-kubelet-config\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159371 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/94e520a8-48a2-4f5e-9dce-f6487ff55798-tmp-dir\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159375 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-run\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.160702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159376 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d4338e08-6387-4995-8195-d0ab26a18e5b-host\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159401 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-cnibin\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159435 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d4338e08-6387-4995-8195-d0ab26a18e5b-serviceca\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159446 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-etc-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159473 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovnkube-config\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159503 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-cni-binary-copy\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159530 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chsjp\" (UniqueName: \"kubernetes.io/projected/982e3dc6-626c-4fd5-9616-6670f54a108c-kube-api-access-chsjp\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159565 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-iptables-alerter-script\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159594 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-cni-binary-copy\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159619 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jltx\" (UniqueName: \"kubernetes.io/projected/94e520a8-48a2-4f5e-9dce-f6487ff55798-kube-api-access-9jltx\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159644 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-slash\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159668 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-log-socket\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-os-release\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159709 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4f6c5d5e-2d4b-4b0b-8eae-12039897236e-konnectivity-ca\") pod \"konnectivity-agent-krqkm\" (UID: \"4f6c5d5e-2d4b-4b0b-8eae-12039897236e\") " pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159717 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-cni-multus\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159742 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-conf-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f6c5d5e-2d4b-4b0b-8eae-12039897236e-agent-certs\") pod \"konnectivity-agent-krqkm\" (UID: \"4f6c5d5e-2d4b-4b0b-8eae-12039897236e\") " pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.161336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159822 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-multus-conf-dir\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159933 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-cni-binary-copy\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.159974 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh7wm\" (UniqueName: \"kubernetes.io/projected/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-kube-api-access-qh7wm\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-host-var-lib-cni-multus\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160040 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-dbus\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160041 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-cni-binary-copy\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160087 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-os-release\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160087 2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160153 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160179 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-etc-kubernetes\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-kubernetes\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160261 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-etc-kubernetes\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160289 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-kubelet\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160317 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovn-node-metrics-cert\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160363 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-device-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160393 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-etc-selinux\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.161877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160418 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-host-slash\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-system-cni-dir\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160492 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53b05de2-d740-42a4-886a-f10ec45dbef5-system-cni-dir\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160495 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-systemd\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160531 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-tuned\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160560 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160616 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovnkube-script-lib\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160645 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-sys-fs\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160672 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-cnibin\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160695 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-var-lib-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160699 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/53b05de2-d740-42a4-886a-f10ec45dbef5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160719 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.162399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.160759 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-cnibin\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.163093 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.163076 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4f6c5d5e-2d4b-4b0b-8eae-12039897236e-agent-certs\") pod \"konnectivity-agent-krqkm\" (UID: \"4f6c5d5e-2d4b-4b0b-8eae-12039897236e\") " pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.173068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.172937 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5pqp\" (UniqueName: \"kubernetes.io/projected/53b05de2-d740-42a4-886a-f10ec45dbef5-kube-api-access-p5pqp\") pod \"multus-additional-cni-plugins-lkm55\" (UID: \"53b05de2-d740-42a4-886a-f10ec45dbef5\") " pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.173068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.173026 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jltx\" (UniqueName: \"kubernetes.io/projected/94e520a8-48a2-4f5e-9dce-f6487ff55798-kube-api-access-9jltx\") pod \"node-resolver-n88h9\" (UID: \"94e520a8-48a2-4f5e-9dce-f6487ff55798\") " pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.173068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.173053 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5w2x\" (UniqueName: \"kubernetes.io/projected/d4338e08-6387-4995-8195-d0ab26a18e5b-kube-api-access-j5w2x\") pod \"node-ca-ddk2j\" (UID: \"d4338e08-6387-4995-8195-d0ab26a18e5b\") " pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.173068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.173069 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh7wm\" (UniqueName: \"kubernetes.io/projected/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-kube-api-access-qh7wm\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.173361 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.173345 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gxs2\" (UniqueName: \"kubernetes.io/projected/fc9d82d3-31c2-48be-9913-7bf5684d1e1f-kube-api-access-8gxs2\") pod \"multus-km9xf\" (UID: \"fc9d82d3-31c2-48be-9913-7bf5684d1e1f\") " pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.243551 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.243516 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf731d16102760bfb05c5d6c7046a2b9d.slice/crio-8f7682ab2ae2022f17ad99d2bbc172246e10c7ab660434fe4477445f8261bc7f WatchSource:0}: Error finding container 8f7682ab2ae2022f17ad99d2bbc172246e10c7ab660434fe4477445f8261bc7f: Status 404 returned error can't find the container with id 8f7682ab2ae2022f17ad99d2bbc172246e10c7ab660434fe4477445f8261bc7f Jul 06 22:24:29.243774 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.243750 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d8e8123e63af15df39698cf832a3600.slice/crio-7ca8445c69ac9d621d434034c5b1ad068d5a97bf3711c3b6d05420c525fe3e1d WatchSource:0}: Error finding container 7ca8445c69ac9d621d434034c5b1ad068d5a97bf3711c3b6d05420c525fe3e1d: Status 404 returned error can't find the container with id 7ca8445c69ac9d621d434034c5b1ad068d5a97bf3711c3b6d05420c525fe3e1d Jul 06 22:24:29.247682 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.247657 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:24:29.261453 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261431 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982e3dc6-626c-4fd5-9616-6670f54a108c-tmp\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.261561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261460 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-registration-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.261561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261477 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-kubelet-config\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.261561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261499 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-run\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.261561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261520 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-etc-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.261561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261544 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovnkube-config\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261567 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-registration-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261578 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-kubelet-config\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261577 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-run\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261602 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-etc-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261641 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chsjp\" (UniqueName: \"kubernetes.io/projected/982e3dc6-626c-4fd5-9616-6670f54a108c-kube-api-access-chsjp\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261664 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-iptables-alerter-script\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261684 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-slash\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261699 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-log-socket\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-dbus\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.261756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261746 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261759 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-slash\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261773 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-log-socket\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.261823 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261873 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-dbus\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.261879 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:29.761860867 +0000 UTC m=+2.066029506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-kubernetes\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261942 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-kubelet\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261968 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-kubernetes\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261978 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovn-node-metrics-cert\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.261989 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-kubelet\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262008 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-device-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262032 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-etc-selinux\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262056 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-host-slash\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262087 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-device-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262103 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-host-slash\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262122 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-systemd\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.262163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262142 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-etc-selinux\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262150 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-tuned\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262175 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262191 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-systemd\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovnkube-script-lib\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-sys-fs\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262246 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-run-ovn-kubernetes\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262274 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-var-lib-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262281 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-iptables-alerter-script\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262295 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-sys-fs\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262299 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262275 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovnkube-config\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262335 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262361 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysctl-d\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262395 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-var-lib-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262418 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-sys\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262439 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-host\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262462 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262479 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysctl-d\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262486 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5g8n\" (UniqueName: \"kubernetes.io/projected/3c5f9cb7-f0f2-470d-811a-6a0015455d42-kube-api-access-j5g8n\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262693 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-host\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262699 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-sys\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262723 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-socket-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-openvswitch\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262767 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-var-lib-kubelet\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262782 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovnkube-script-lib\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262796 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-run-netns\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262831 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-run-netns\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262834 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-systemd\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262853 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-var-lib-kubelet\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262872 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-env-overrides\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262854 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/64fbf0a0-0f71-4641-ac1f-b88539056744-socket-dir\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262875 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-systemd\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262901 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pf65\" (UniqueName: \"kubernetes.io/projected/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-kube-api-access-9pf65\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.263993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262931 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-modprobe-d\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262956 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-lib-modules\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.262982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-cni-netd\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263010 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysctl-conf\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263033 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-systemd-units\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263057 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-ovn\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263087 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-node-log\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263141 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263155 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-cni-netd\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263165 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263194 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3c5f9cb7-f0f2-470d-811a-6a0015455d42-env-overrides\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263198 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw6zw\" (UniqueName: \"kubernetes.io/projected/64fbf0a0-0f71-4641-ac1f-b88539056744-kube-api-access-xw6zw\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263252 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-modprobe-d\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263271 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-cni-bin\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263299 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysconfig\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263325 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-lib-modules\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263376 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysconfig\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.264695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263371 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-run-ovn\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263418 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-cni-bin\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263463 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-systemd-units\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263488 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-sysctl-conf\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263494 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-node-log\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.263531 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3c5f9cb7-f0f2-470d-811a-6a0015455d42-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.264167 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/982e3dc6-626c-4fd5-9616-6670f54a108c-tmp\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.264369 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3c5f9cb7-f0f2-470d-811a-6a0015455d42-ovn-node-metrics-cert\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.265148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.264590 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/982e3dc6-626c-4fd5-9616-6670f54a108c-etc-tuned\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.270490 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.270463 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:24:29.270490 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.270487 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:24:29.270666 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.270501 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:29.270666 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.270561 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:29.770543797 +0000 UTC m=+2.074712452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:29.271499 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.271473 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chsjp\" (UniqueName: \"kubernetes.io/projected/982e3dc6-626c-4fd5-9616-6670f54a108c-kube-api-access-chsjp\") pod \"tuned-2cg9w\" (UID: \"982e3dc6-626c-4fd5-9616-6670f54a108c\") " pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.272538 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.272515 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pf65\" (UniqueName: \"kubernetes.io/projected/b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2-kube-api-access-9pf65\") pod \"iptables-alerter-q5whp\" (UID: \"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2\") " pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.272701 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.272683 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5g8n\" (UniqueName: \"kubernetes.io/projected/3c5f9cb7-f0f2-470d-811a-6a0015455d42-kube-api-access-j5g8n\") pod \"ovnkube-node-7tdp7\" (UID: \"3c5f9cb7-f0f2-470d-811a-6a0015455d42\") " pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.272957 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.272942 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw6zw\" (UniqueName: \"kubernetes.io/projected/64fbf0a0-0f71-4641-ac1f-b88539056744-kube-api-access-xw6zw\") pod \"aws-ebs-csi-driver-node-lclgr\" (UID: \"64fbf0a0-0f71-4641-ac1f-b88539056744\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.367759 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.367692 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n88h9" Jul 06 22:24:29.369169 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.369148 2572 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:29.375635 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.375606 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94e520a8_48a2_4f5e_9dce_f6487ff55798.slice/crio-66b33bf24c3024e97dd6fbd4edee02621b8f9be64d73cdaf51248f97610e5d37 WatchSource:0}: Error finding container 66b33bf24c3024e97dd6fbd4edee02621b8f9be64d73cdaf51248f97610e5d37: Status 404 returned error can't find the container with id 66b33bf24c3024e97dd6fbd4edee02621b8f9be64d73cdaf51248f97610e5d37 Jul 06 22:24:29.387309 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.387286 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-ddk2j" Jul 06 22:24:29.394539 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.394517 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4338e08_6387_4995_8195_d0ab26a18e5b.slice/crio-338e8e3c4e9cf47e97d44565503383d31e22df54e031e189d266e2700d7186d8 WatchSource:0}: Error finding container 338e8e3c4e9cf47e97d44565503383d31e22df54e031e189d266e2700d7186d8: Status 404 returned error can't find the container with id 338e8e3c4e9cf47e97d44565503383d31e22df54e031e189d266e2700d7186d8 Jul 06 22:24:29.406605 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.406586 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-lkm55" Jul 06 22:24:29.414247 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.414204 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b05de2_d740_42a4_886a_f10ec45dbef5.slice/crio-0b4e8796b62c3adfedd1fa5df1de88200b982648670bd877e310edcf095813c6 WatchSource:0}: Error finding container 0b4e8796b62c3adfedd1fa5df1de88200b982648670bd877e310edcf095813c6: Status 404 returned error can't find the container with id 0b4e8796b62c3adfedd1fa5df1de88200b982648670bd877e310edcf095813c6 Jul 06 22:24:29.419218 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.419202 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-km9xf" Jul 06 22:24:29.425606 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.425585 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:29.425837 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.425817 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc9d82d3_31c2_48be_9913_7bf5684d1e1f.slice/crio-05ec074ceb2f647cbdaa76bca0ab681e1e643774a592d3561c532f7848423b65 WatchSource:0}: Error finding container 05ec074ceb2f647cbdaa76bca0ab681e1e643774a592d3561c532f7848423b65: Status 404 returned error can't find the container with id 05ec074ceb2f647cbdaa76bca0ab681e1e643774a592d3561c532f7848423b65 Jul 06 22:24:29.432220 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.432198 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f6c5d5e_2d4b_4b0b_8eae_12039897236e.slice/crio-f1538dde4ee127fc4f645abb87d7834e01a28a734fae65a6eb035ce58c565142 WatchSource:0}: Error finding container f1538dde4ee127fc4f645abb87d7834e01a28a734fae65a6eb035ce58c565142: Status 404 returned error can't find the container with id f1538dde4ee127fc4f645abb87d7834e01a28a734fae65a6eb035ce58c565142 Jul 06 22:24:29.443000 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.442980 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-q5whp" Jul 06 22:24:29.450467 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.450442 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c14bc9_aeb1_4a46_b268_fdd5c61b6bd2.slice/crio-d6c2ba51f51064e712b5617d90c4f7e979cbcd4533658362ac91a56c558709be WatchSource:0}: Error finding container d6c2ba51f51064e712b5617d90c4f7e979cbcd4533658362ac91a56c558709be: Status 404 returned error can't find the container with id d6c2ba51f51064e712b5617d90c4f7e979cbcd4533658362ac91a56c558709be Jul 06 22:24:29.460475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.460459 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:29.468270 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.468251 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5f9cb7_f0f2_470d_811a_6a0015455d42.slice/crio-a03166bf596d6ca83b65b4d0c01e8b0cd61c63afbc3cbfb2a5af4a4b842213ac WatchSource:0}: Error finding container a03166bf596d6ca83b65b4d0c01e8b0cd61c63afbc3cbfb2a5af4a4b842213ac: Status 404 returned error can't find the container with id a03166bf596d6ca83b65b4d0c01e8b0cd61c63afbc3cbfb2a5af4a4b842213ac Jul 06 22:24:29.485562 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.485543 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" Jul 06 22:24:29.490834 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.490817 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" Jul 06 22:24:29.494077 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.494049 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64fbf0a0_0f71_4641_ac1f_b88539056744.slice/crio-64b4fefa25567e86d7230f20e7eb886ba6d05fd8da66a3b4e7bd5fbc72f85594 WatchSource:0}: Error finding container 64b4fefa25567e86d7230f20e7eb886ba6d05fd8da66a3b4e7bd5fbc72f85594: Status 404 returned error can't find the container with id 64b4fefa25567e86d7230f20e7eb886ba6d05fd8da66a3b4e7bd5fbc72f85594 Jul 06 22:24:29.497950 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:24:29.497932 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod982e3dc6_626c_4fd5_9616_6670f54a108c.slice/crio-f3047b4bf44b1b5ffe70a9dcf02fc4c1ca157dc98b1c36e38922e2d9ee8333c5 WatchSource:0}: Error finding container f3047b4bf44b1b5ffe70a9dcf02fc4c1ca157dc98b1c36e38922e2d9ee8333c5: Status 404 returned error can't find the container with id f3047b4bf44b1b5ffe70a9dcf02fc4c1ca157dc98b1c36e38922e2d9ee8333c5 Jul 06 22:24:29.666223 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.666151 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:29.666377 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.666359 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:29.666428 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.666417 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:30.666400356 +0000 UTC m=+2.970569004 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:29.767249 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.767197 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:29.767412 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.767392 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:29.767481 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.767450 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:30.767433018 +0000 UTC m=+3.071601673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:29.868480 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.868446 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:29.868658 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.868598 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:24:29.868658 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.868620 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:24:29.868658 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.868633 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:29.868812 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:29.868686 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:30.868669579 +0000 UTC m=+3.172838229 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:29.932946 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:29.932917 2572 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:30.094718 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.094434 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:19:29 +0000 UTC" deadline="2028-01-19 15:18:23.410920781 +0000 UTC" Jul 06 22:24:30.094718 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.094498 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13480h53m53.316427056s" Jul 06 22:24:30.265363 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.262873 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"a03166bf596d6ca83b65b4d0c01e8b0cd61c63afbc3cbfb2a5af4a4b842213ac"} Jul 06 22:24:30.269307 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.269228 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q5whp" event={"ID":"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2","Type":"ContainerStarted","Data":"d6c2ba51f51064e712b5617d90c4f7e979cbcd4533658362ac91a56c558709be"} Jul 06 22:24:30.274518 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.274489 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-km9xf" event={"ID":"fc9d82d3-31c2-48be-9913-7bf5684d1e1f","Type":"ContainerStarted","Data":"05ec074ceb2f647cbdaa76bca0ab681e1e643774a592d3561c532f7848423b65"} Jul 06 22:24:30.303397 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.303368 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n88h9" event={"ID":"94e520a8-48a2-4f5e-9dce-f6487ff55798","Type":"ContainerStarted","Data":"66b33bf24c3024e97dd6fbd4edee02621b8f9be64d73cdaf51248f97610e5d37"} Jul 06 22:24:30.320620 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.320590 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" event={"ID":"4d8e8123e63af15df39698cf832a3600","Type":"ContainerStarted","Data":"7ca8445c69ac9d621d434034c5b1ad068d5a97bf3711c3b6d05420c525fe3e1d"} Jul 06 22:24:30.341564 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.341529 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krqkm" event={"ID":"4f6c5d5e-2d4b-4b0b-8eae-12039897236e","Type":"ContainerStarted","Data":"f1538dde4ee127fc4f645abb87d7834e01a28a734fae65a6eb035ce58c565142"} Jul 06 22:24:30.356078 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.356007 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerStarted","Data":"0b4e8796b62c3adfedd1fa5df1de88200b982648670bd877e310edcf095813c6"} Jul 06 22:24:30.374401 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.374330 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ddk2j" event={"ID":"d4338e08-6387-4995-8195-d0ab26a18e5b","Type":"ContainerStarted","Data":"338e8e3c4e9cf47e97d44565503383d31e22df54e031e189d266e2700d7186d8"} Jul 06 22:24:30.389832 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.389750 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" event={"ID":"f731d16102760bfb05c5d6c7046a2b9d","Type":"ContainerStarted","Data":"8f7682ab2ae2022f17ad99d2bbc172246e10c7ab660434fe4477445f8261bc7f"} Jul 06 22:24:30.398407 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.398339 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" event={"ID":"982e3dc6-626c-4fd5-9616-6670f54a108c","Type":"ContainerStarted","Data":"f3047b4bf44b1b5ffe70a9dcf02fc4c1ca157dc98b1c36e38922e2d9ee8333c5"} Jul 06 22:24:30.420825 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.420797 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" event={"ID":"64fbf0a0-0f71-4641-ac1f-b88539056744","Type":"ContainerStarted","Data":"64b4fefa25567e86d7230f20e7eb886ba6d05fd8da66a3b4e7bd5fbc72f85594"} Jul 06 22:24:30.676204 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.675687 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:30.676204 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.675923 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:30.676204 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.675968 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:32.675954716 +0000 UTC m=+4.980123353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:30.776632 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.776595 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:30.776810 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.776793 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:30.776889 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.776861 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:32.77684132 +0000 UTC m=+5.081009971 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:30.877285 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:30.877248 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:30.877446 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.877409 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:24:30.877446 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.877429 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:24:30.877446 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.877443 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:30.877621 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:30.877505 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:32.877483927 +0000 UTC m=+5.181652572 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:31.009768 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.009489 2572 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:31.095324 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.095284 2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:19:29 +0000 UTC" deadline="2028-02-22 06:49:14.283756498 +0000 UTC" Jul 06 22:24:31.095324 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.095319 2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14288h24m43.188442085s" Jul 06 22:24:31.242545 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.242473 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:31.242689 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:31.242620 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:31.243075 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.243053 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:31.243177 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:31.243155 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:31.243266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.243257 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:31.243370 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:31.243350 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:31.464581 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:31.464551 2572 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:24:32.690119 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:32.690080 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:32.690636 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.690248 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:32.690636 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.690311 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:36.690292623 +0000 UTC m=+8.994461276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:32.790802 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:32.790751 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:32.790976 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.790905 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:32.790976 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.790967 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:36.790950237 +0000 UTC m=+9.095118880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:32.891201 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:32.891158 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:32.891430 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.891340 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:24:32.891430 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.891358 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:24:32.891430 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.891369 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:32.891430 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:32.891427 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:36.891406577 +0000 UTC m=+9.195575232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:33.242481 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:33.242447 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:33.242818 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:33.242561 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:33.242929 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:33.242910 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:33.243006 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:33.242988 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:33.243058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:33.243048 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:33.243124 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:33.243109 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:35.242797 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:35.242758 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:35.242797 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:35.242794 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:35.243182 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:35.242758 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:35.243182 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:35.242885 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:35.243182 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:35.242938 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:35.243182 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:35.243010 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:36.721765 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:36.721726 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:36.722288 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.721918 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:36.722288 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.721975 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:24:44.721960587 +0000 UTC m=+17.026129225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:36.822855 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:36.822814 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:36.823061 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.822939 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:36.823061 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.822996 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:44.822977679 +0000 UTC m=+17.127146333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:36.924411 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:36.923813 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:36.924411 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.923969 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:24:36.924411 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.923996 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:24:36.924411 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.924005 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:36.924411 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:36.924053 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:24:44.924037913 +0000 UTC m=+17.228206551 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:37.241863 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:37.241825 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:37.242028 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:37.241871 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:37.242071 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:37.242031 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:37.242071 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:37.241972 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:37.242140 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:37.242088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:37.242196 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:37.242172 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:39.242315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:39.242266 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:39.242775 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:39.242265 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:39.242775 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:39.242389 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:39.242775 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:39.242458 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:39.242775 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:39.242266 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:39.242775 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:39.242556 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:41.242441 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:41.242401 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:41.242858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:41.242405 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:41.242858 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:41.242539 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:41.242858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:41.242549 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:41.242858 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:41.242627 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:41.242858 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:41.242706 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:43.242033 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:43.241993 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:43.242523 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:43.242043 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:43.242523 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:43.242109 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:43.242523 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:43.242127 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:43.242523 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:43.242202 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:43.242523 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:43.242305 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:44.785132 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:44.785095 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:44.785616 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.785248 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:44.785616 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.785311 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:25:00.78529511 +0000 UTC m=+33.089463753 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:24:44.886017 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:44.885979 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:44.886194 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.886102 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:44.886194 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.886166 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:00.886147823 +0000 UTC m=+33.190316460 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:24:44.986852 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:44.986812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:44.987018 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.986970 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:24:44.987018 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.986994 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:24:44.987018 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.987004 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:44.987161 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:44.987071 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:00.987052045 +0000 UTC m=+33.291220685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:24:45.242585 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:45.242524 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:45.242585 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:45.242572 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:45.242806 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:45.242629 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:45.242806 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:45.242646 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:45.242806 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:45.242728 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:45.242806 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:45.242776 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:47.242754 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:47.242716 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:47.243175 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:47.242765 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:47.243175 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:47.242765 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:47.243175 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:47.242841 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:47.243175 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:47.242980 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:47.243175 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:47.243079 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:49.242467 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:49.242432 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:49.242884 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:49.242467 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:49.242884 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:49.242567 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:49.242884 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:49.242585 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:49.242884 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:49.242632 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:49.242884 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:49.242714 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:50.470324 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.470284 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" event={"ID":"982e3dc6-626c-4fd5-9616-6670f54a108c","Type":"ContainerStarted","Data":"b50be5b6e92a80f6b1bc4e8738ccb1930ccae754471dff97f9cb8c3d171ba64a"} Jul 06 22:24:50.473615 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.473596 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:24:50.474340 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474319 2572 generic.go:356] "Generic (PLEG): container finished" podID="3c5f9cb7-f0f2-470d-811a-6a0015455d42" containerID="4632e2ace443b642ff1afbb3aedee518f91629ceae2051eb368e51c9f9c39848" exitCode=1 Jul 06 22:24:50.474432 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474394 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"5fda8dfd69637419693675a88c6501dff5926145218d1145abfefe74b6e41de6"} Jul 06 22:24:50.474500 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474483 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"9f9a608d6d8f3f7a45ec025d0a0e058a26c2d5431222b9fc48479fcabc3b2157"} Jul 06 22:24:50.474552 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474540 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"430bb4e8c80893b428a41bfb9f19a4eed9a0549f9a9e8aa5d1d00c96623a78b5"} Jul 06 22:24:50.474600 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474586 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"c1d4089003f2c32df73d3f4757e43081fb0ae3c8073ca4ca20f232b37963001f"} Jul 06 22:24:50.474648 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474630 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerDied","Data":"4632e2ace443b642ff1afbb3aedee518f91629ceae2051eb368e51c9f9c39848"} Jul 06 22:24:50.474726 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.474683 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"38d93801ec314857e91a4d3b82a5fabd60a9ae131c9a6de53b8ca298eea14de3"} Jul 06 22:24:50.476883 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.476844 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-km9xf" event={"ID":"fc9d82d3-31c2-48be-9913-7bf5684d1e1f","Type":"ContainerStarted","Data":"a32e095dbf57b9005c6ef3f540e5ba91c7b2f9c436e97db464a84e2fab6f43e9"} Jul 06 22:24:50.478912 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.478548 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" event={"ID":"4d8e8123e63af15df39698cf832a3600","Type":"ContainerStarted","Data":"2b05f42c0711e55272c74f3d8884b54a39e37c15968475691e3c28cdc0264853"} Jul 06 22:24:50.492198 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.491736 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2cg9w" podStartSLOduration=2.327574374 podStartE2EDuration="22.491722689s" totalImagesPullingTime="20.164148315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:50.491641462 +0000 UTC m=+22.795810150" watchObservedRunningTime="2026-07-06 22:24:50.491722689 +0000 UTC m=+22.795891346" Jul 06 22:24:50.507943 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.507908 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-23.ec2.internal" podStartSLOduration=22.507896535 podStartE2EDuration="22.507896535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:50.507457972 +0000 UTC m=+22.811626629" watchObservedRunningTime="2026-07-06 22:24:50.507896535 +0000 UTC m=+22.812065203" Jul 06 22:24:50.526463 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:50.526426 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-km9xf" podStartSLOduration=1.90715573 podStartE2EDuration="22.526412005s" totalImagesPullingTime="20.619256275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:50.525410908 +0000 UTC m=+22.829579567" watchObservedRunningTime="2026-07-06 22:24:50.526412005 +0000 UTC m=+22.830580650" Jul 06 22:24:51.241791 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.241709 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:51.242034 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.241714 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:51.242034 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:51.241799 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:51.242034 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.241819 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:51.242034 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:51.241897 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:51.242034 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:51.241962 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:51.482442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.482406 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" event={"ID":"64fbf0a0-0f71-4641-ac1f-b88539056744","Type":"ContainerStarted","Data":"47ba1028855061b613fd299055d8d818035990380aaf37b8cfe0a7b2d968eec7"} Jul 06 22:24:51.483771 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.483741 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-q5whp" event={"ID":"b3c14bc9-aeb1-4a46-b268-fdd5c61b6bd2","Type":"ContainerStarted","Data":"3b57878f27572f5c0dbe59849e63205e2982fb1c9e6ac369a69907aa3d8e1ef6"} Jul 06 22:24:51.486124 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.486096 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n88h9" event={"ID":"94e520a8-48a2-4f5e-9dce-f6487ff55798","Type":"ContainerStarted","Data":"17b6cb3f12428f550611d10d6f9252ff6da09ed78941ff149e6dab75c871d456"} Jul 06 22:24:51.487577 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.487545 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-krqkm" event={"ID":"4f6c5d5e-2d4b-4b0b-8eae-12039897236e","Type":"ContainerStarted","Data":"b49cae8af2b162dee6d15c70fd263a14161321404e694bea638ad24990d4affa"} Jul 06 22:24:51.489421 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.489394 2572 generic.go:356] "Generic (PLEG): container finished" podID="53b05de2-d740-42a4-886a-f10ec45dbef5" containerID="dbc53bec51bccc0d19b0ab60945095f06404b77d3148796404b2b99e77731730" exitCode=0 Jul 06 22:24:51.489529 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.489485 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerDied","Data":"dbc53bec51bccc0d19b0ab60945095f06404b77d3148796404b2b99e77731730"} Jul 06 22:24:51.490957 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.490940 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-ddk2j" event={"ID":"d4338e08-6387-4995-8195-d0ab26a18e5b","Type":"ContainerStarted","Data":"2d4a4f1883f97dc2d3e72c9db145ca9bd49de2a8ea27577ece591446af4f8950"} Jul 06 22:24:51.492412 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.492332 2572 generic.go:356] "Generic (PLEG): container finished" podID="f731d16102760bfb05c5d6c7046a2b9d" containerID="81ef0ffe311e7be39ddc01de73330a5bd6c63d707388eff3767623f94ade97ac" exitCode=0 Jul 06 22:24:51.492500 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.492411 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" event={"ID":"f731d16102760bfb05c5d6c7046a2b9d","Type":"ContainerDied","Data":"81ef0ffe311e7be39ddc01de73330a5bd6c63d707388eff3767623f94ade97ac"} Jul 06 22:24:51.520859 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.520810 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-q5whp" podStartSLOduration=3.3449506749999998 podStartE2EDuration="23.520793863s" totalImagesPullingTime="20.175843188s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:51.520629458 +0000 UTC m=+23.824798116" watchObservedRunningTime="2026-07-06 22:24:51.520793863 +0000 UTC m=+23.824962522" Jul 06 22:24:51.632076 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.632031 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-krqkm" podStartSLOduration=3.40518891 podStartE2EDuration="23.632013286s" totalImagesPullingTime="20.226824376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:51.591580196 +0000 UTC m=+23.895748855" watchObservedRunningTime="2026-07-06 22:24:51.632013286 +0000 UTC m=+23.936181945" Jul 06 22:24:51.689720 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.689653 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-n88h9" podStartSLOduration=3.4242837919999998 podStartE2EDuration="23.689636789s" totalImagesPullingTime="20.265352997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:51.632146483 +0000 UTC m=+23.936315127" watchObservedRunningTime="2026-07-06 22:24:51.689636789 +0000 UTC m=+23.993805454" Jul 06 22:24:51.689882 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:51.689747 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-ddk2j" podStartSLOduration=3.458273571 podStartE2EDuration="23.689740643s" totalImagesPullingTime="20.231467072s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:51.686303942 +0000 UTC m=+23.990472602" watchObservedRunningTime="2026-07-06 22:24:51.689740643 +0000 UTC m=+23.993909300" Jul 06 22:24:52.131951 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:52.131914 2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 22:24:52.137543 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:52.137458 2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T22:24:52.131932271Z","UUID":"1af37be9-4e86-485c-a53d-b69ae5db11f2","Handler":null,"Name":"","Endpoint":""} Jul 06 22:24:52.138878 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:52.138861 2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 22:24:52.138970 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:52.138902 2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 22:24:52.497361 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:52.497299 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" event={"ID":"64fbf0a0-0f71-4641-ac1f-b88539056744","Type":"ContainerStarted","Data":"646ac27ec2da841a1d2974dcfc18ce706117ca9a93c19d6c88450727b7fc334b"} Jul 06 22:24:53.242552 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.241852 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:53.242552 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.241892 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:53.242552 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:53.242187 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:53.242552 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:53.242274 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:53.242552 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.241892 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:53.242552 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:53.242363 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:53.501726 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.501690 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" event={"ID":"f731d16102760bfb05c5d6c7046a2b9d","Type":"ContainerStarted","Data":"512f44e039d8a5357cbc89987d734e18f786765b1401bf20029a8156aad5ca0b"} Jul 06 22:24:53.504126 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.504090 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" event={"ID":"64fbf0a0-0f71-4641-ac1f-b88539056744","Type":"ContainerStarted","Data":"7a8758b77c2654155df9f7337af00fcfe36370e488134a68dd275edb15a2398b"} Jul 06 22:24:53.509631 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.509612 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:24:53.509989 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.509963 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"0288905ff505e25fff948c9831edc89db7d244da17946e065beaf9d32655d047"} Jul 06 22:24:53.520490 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.520445 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-23.ec2.internal" podStartSLOduration=25.52043351 podStartE2EDuration="25.52043351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:53.520134769 +0000 UTC m=+25.824303426" watchObservedRunningTime="2026-07-06 22:24:53.52043351 +0000 UTC m=+25.824602169" Jul 06 22:24:53.537908 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.537863 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lclgr" podStartSLOduration=3.500589681 podStartE2EDuration="25.537852895s" totalImagesPullingTime="22.037263214s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:53.537477819 +0000 UTC m=+25.841646478" watchObservedRunningTime="2026-07-06 22:24:53.537852895 +0000 UTC m=+25.842021554" Jul 06 22:24:53.621874 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.621844 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:53.622832 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:53.622813 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:54.512573 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:54.512538 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:54.513123 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:54.513017 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-krqkm" Jul 06 22:24:55.242058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:55.242017 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:55.242250 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:55.242135 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:55.242250 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:55.242147 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:55.242250 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:55.242138 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:55.242436 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:55.242249 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:55.242436 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:55.242327 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:56.518479 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.518303 2572 generic.go:356] "Generic (PLEG): container finished" podID="53b05de2-d740-42a4-886a-f10ec45dbef5" containerID="fa6c10ab267fb588731cb83dd8a5448737b5fbb0fbb277255979110429007e3f" exitCode=0 Jul 06 22:24:56.519176 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.518381 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerDied","Data":"fa6c10ab267fb588731cb83dd8a5448737b5fbb0fbb277255979110429007e3f"} Jul 06 22:24:56.522025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.522007 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:24:56.522392 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.522372 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"2c1a9603a8d63b45f012025198c86c99d3f9697af2cb2282b1494dad33d37661"} Jul 06 22:24:56.522625 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.522608 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:56.522737 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.522723 2572 scope.go:117] "RemoveContainer" containerID="4632e2ace443b642ff1afbb3aedee518f91629ceae2051eb368e51c9f9c39848" Jul 06 22:24:56.538579 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:56.538557 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:57.242510 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.242476 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:57.242745 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.242515 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:57.242745 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.242596 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:57.242745 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:57.242609 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:24:57.242745 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:57.242706 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:57.242948 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:57.242762 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:57.527845 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.527780 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:24:57.528262 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.528116 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" event={"ID":"3c5f9cb7-f0f2-470d-811a-6a0015455d42","Type":"ContainerStarted","Data":"2eaf5c8db7c86db2d3e9a74d774b68f27e8499cf16227330a54ddd42d20a568f"} Jul 06 22:24:57.528359 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.528306 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:57.528413 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.528364 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:57.543965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.543944 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:24:57.556520 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:57.556475 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" podStartSLOduration=9.324120366 podStartE2EDuration="29.556462689s" totalImagesPullingTime="20.232342323s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:24:57.555869265 +0000 UTC m=+29.860037946" watchObservedRunningTime="2026-07-06 22:24:57.556462689 +0000 UTC m=+29.860631347" Jul 06 22:24:58.532582 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:58.532549 2572 generic.go:356] "Generic (PLEG): container finished" podID="53b05de2-d740-42a4-886a-f10ec45dbef5" containerID="f64803d1866ce018b316a8faa18a29ca90c09f1563849689b0691974c20eb70a" exitCode=0 Jul 06 22:24:58.532996 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:58.532643 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerDied","Data":"f64803d1866ce018b316a8faa18a29ca90c09f1563849689b0691974c20eb70a"} Jul 06 22:24:59.242732 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:59.242696 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:24:59.242902 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:59.242736 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:24:59.242902 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:24:59.242742 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:24:59.242902 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:59.242820 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:24:59.242902 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:59.242888 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:24:59.243106 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:24:59.242960 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:00.539014 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:00.538839 2572 generic.go:356] "Generic (PLEG): container finished" podID="53b05de2-d740-42a4-886a-f10ec45dbef5" containerID="37662940830f81775cfdda694519f18a8426b19d351805836106efd7cf32c82b" exitCode=0 Jul 06 22:25:00.539014 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:00.538927 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerDied","Data":"37662940830f81775cfdda694519f18a8426b19d351805836106efd7cf32c82b"} Jul 06 22:25:00.809273 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:00.809187 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:00.809402 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:00.809306 2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:25:00.809402 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:00.809358 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs podName:158c4ff9-36f2-454d-8082-94ab9b2e5bdb nodeName:}" failed. No retries permitted until 2026-07-06 22:25:32.809345032 +0000 UTC m=+65.113513673 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs") pod "network-metrics-daemon-2sj22" (UID: "158c4ff9-36f2-454d-8082-94ab9b2e5bdb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:25:00.910034 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:00.909995 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:00.910178 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:00.910130 2572 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:25:00.910215 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:00.910182 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret podName:8b4cf0a7-678c-4f21-8d45-a46b9e1dd522 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:32.910168305 +0000 UTC m=+65.214336943 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret") pod "global-pull-secret-syncer-vlvg2" (UID: "8b4cf0a7-678c-4f21-8d45-a46b9e1dd522") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:25:01.010694 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:01.010661 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:01.010855 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.010778 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:25:01.010855 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.010790 2572 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:25:01.010855 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.010801 2572 projected.go:196] Error preparing data for projected volume kube-api-access-jrvjf for pod openshift-network-diagnostics/network-check-target-xrmgf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:25:01.010855 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.010845 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf podName:361ac06c-1abe-49fd-bd1a-4f8083a9e962 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:33.010833238 +0000 UTC m=+65.315001874 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jrvjf" (UniqueName: "kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf") pod "network-check-target-xrmgf" (UID: "361ac06c-1abe-49fd-bd1a-4f8083a9e962") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:25:01.242019 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:01.241987 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:01.242208 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.242103 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:01.242208 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:01.241990 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:01.242208 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:01.241990 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:01.242208 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.242190 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:01.242405 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:01.242280 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:03.242571 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:03.242536 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:03.242955 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:03.242583 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:03.242955 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:03.242643 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:03.242955 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:03.242662 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:03.242955 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:03.242718 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:03.242955 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:03.242812 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:05.241683 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:05.241656 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:05.242033 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:05.241750 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:05.242033 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:05.241755 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:05.242033 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:05.241769 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:05.242033 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:05.241842 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:05.242033 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:05.241889 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:05.552972 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:05.552884 2572 generic.go:356] "Generic (PLEG): container finished" podID="53b05de2-d740-42a4-886a-f10ec45dbef5" containerID="2092ef66ccaf193de6caf90e0310eee4dfd40c33a5d7eb7b798c4c02bfe9303f" exitCode=0 Jul 06 22:25:05.552972 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:05.552930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerDied","Data":"2092ef66ccaf193de6caf90e0310eee4dfd40c33a5d7eb7b798c4c02bfe9303f"} Jul 06 22:25:06.557942 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:06.557909 2572 generic.go:356] "Generic (PLEG): container finished" podID="53b05de2-d740-42a4-886a-f10ec45dbef5" containerID="7621ddb63d6112d5077afc0cc731d226bf72245da71dd3ba0169f7f5377afb26" exitCode=0 Jul 06 22:25:06.558491 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:06.557951 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerDied","Data":"7621ddb63d6112d5077afc0cc731d226bf72245da71dd3ba0169f7f5377afb26"} Jul 06 22:25:07.242042 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:07.242002 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:07.242224 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:07.242054 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:07.242224 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:07.242108 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:07.242224 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:07.242119 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:07.242224 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:07.242171 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:07.242399 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:07.242272 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:07.562857 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:07.562779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-lkm55" event={"ID":"53b05de2-d740-42a4-886a-f10ec45dbef5","Type":"ContainerStarted","Data":"84771beffc7a2f2148b5ad9680281b0a70865735b49cd4744020f7c7125f503d"} Jul 06 22:25:07.590749 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:07.590699 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-lkm55" podStartSLOduration=8.712279251 podStartE2EDuration="39.590686927s" totalImagesPullingTime="30.878407676s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:07.590390152 +0000 UTC m=+39.894558809" watchObservedRunningTime="2026-07-06 22:25:07.590686927 +0000 UTC m=+39.894855584" Jul 06 22:25:09.242456 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:09.242428 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:09.242831 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:09.242493 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:09.242831 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:09.242508 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:09.242831 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:09.242593 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:09.242831 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:09.242645 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:09.242831 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:09.242752 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:10.809741 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:10.809505 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlvg2"] Jul 06 22:25:10.810389 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:10.809833 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:10.810389 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:10.809950 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:10.812776 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:10.812744 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xrmgf"] Jul 06 22:25:10.812915 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:10.812860 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:10.812988 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:10.812955 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:10.813546 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:10.813526 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2sj22"] Jul 06 22:25:10.813623 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:10.813607 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:10.813725 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:10.813704 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:12.241689 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:12.241664 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:12.242059 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:12.241783 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:12.242059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:12.241834 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:12.242059 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:12.241915 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:13.242145 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:13.242113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:13.242517 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:13.242222 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-vlvg2" podUID="8b4cf0a7-678c-4f21-8d45-a46b9e1dd522" Jul 06 22:25:14.242022 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:14.241985 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:14.242022 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:14.242030 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:14.242702 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:14.242124 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2sj22" podUID="158c4ff9-36f2-454d-8082-94ab9b2e5bdb" Jul 06 22:25:14.242702 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:14.242301 2572 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xrmgf" podUID="361ac06c-1abe-49fd-bd1a-4f8083a9e962" Jul 06 22:25:14.989989 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:14.989961 2572 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-23.ec2.internal" event="NodeReady" Jul 06 22:25:14.990153 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:14.990106 2572 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 22:25:15.042440 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.042406 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-j4mg6"] Jul 06 22:25:15.063446 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.063420 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbqm9"] Jul 06 22:25:15.063584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.063549 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.066681 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.066658 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 22:25:15.066681 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.066676 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 22:25:15.066864 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.066688 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 22:25:15.066864 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.066752 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bgz5b\"" Jul 06 22:25:15.066864 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.066658 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 22:25:15.087462 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.087441 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j4mg6"] Jul 06 22:25:15.087597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.087478 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbqm9"] Jul 06 22:25:15.087597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.087551 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.090905 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.090885 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 22:25:15.090905 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.090885 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 22:25:15.091086 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.090913 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 22:25:15.091086 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.090885 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zvrrt\"" Jul 06 22:25:15.138476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.138449 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2hg7t"] Jul 06 22:25:15.153558 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.153529 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2hg7t"] Jul 06 22:25:15.153715 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.153625 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.156571 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.156542 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 22:25:15.156693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.156570 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mrg4w\"" Jul 06 22:25:15.156693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.156630 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 22:25:15.216055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216029 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6125683-3f5d-4cc6-ad3d-84b96bae4530-cert\") pod \"ingress-canary-gbqm9\" (UID: \"e6125683-3f5d-4cc6-ad3d-84b96bae4530\") " pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.216055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216057 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9hns\" (UniqueName: \"kubernetes.io/projected/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-kube-api-access-q9hns\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.216263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216079 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-data-volume\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.216263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-crio-socket\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.216263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216174 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.216263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216205 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.216263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.216227 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx9gn\" (UniqueName: \"kubernetes.io/projected/e6125683-3f5d-4cc6-ad3d-84b96bae4530-kube-api-access-xx9gn\") pod \"ingress-canary-gbqm9\" (UID: \"e6125683-3f5d-4cc6-ad3d-84b96bae4530\") " pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.242288 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.242213 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:15.244919 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.244905 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:25:15.316842 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.316801 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.316856 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx9gn\" (UniqueName: \"kubernetes.io/projected/e6125683-3f5d-4cc6-ad3d-84b96bae4530-kube-api-access-xx9gn\") pod \"ingress-canary-gbqm9\" (UID: \"e6125683-3f5d-4cc6-ad3d-84b96bae4530\") " pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.317015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.316893 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-metrics-tls\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.317015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.316940 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6125683-3f5d-4cc6-ad3d-84b96bae4530-cert\") pod \"ingress-canary-gbqm9\" (UID: \"e6125683-3f5d-4cc6-ad3d-84b96bae4530\") " pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.317015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.316964 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9hns\" (UniqueName: \"kubernetes.io/projected/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-kube-api-access-q9hns\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.316987 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-tmp-dir\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.317015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317012 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-config-volume\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.317509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317136 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-data-volume\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317166 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-crio-socket\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317222 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317266 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grbqn\" (UniqueName: \"kubernetes.io/projected/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-kube-api-access-grbqn\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.317509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317458 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-data-volume\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317498 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-crio-socket\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.317924 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.317907 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.320905 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.320886 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.321044 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.321027 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e6125683-3f5d-4cc6-ad3d-84b96bae4530-cert\") pod \"ingress-canary-gbqm9\" (UID: \"e6125683-3f5d-4cc6-ad3d-84b96bae4530\") " pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.326072 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.326051 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx9gn\" (UniqueName: \"kubernetes.io/projected/e6125683-3f5d-4cc6-ad3d-84b96bae4530-kube-api-access-xx9gn\") pod \"ingress-canary-gbqm9\" (UID: \"e6125683-3f5d-4cc6-ad3d-84b96bae4530\") " pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.326269 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.326227 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9hns\" (UniqueName: \"kubernetes.io/projected/7fc4d630-709f-4f3e-a3ff-4b93fde2bb80-kube-api-access-q9hns\") pod \"insights-runtime-extractor-j4mg6\" (UID: \"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80\") " pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.373175 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.373137 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-j4mg6" Jul 06 22:25:15.396289 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.396262 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbqm9" Jul 06 22:25:15.418419 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.418395 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-tmp-dir\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.418531 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.418426 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-config-volume\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.418531 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.418471 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grbqn\" (UniqueName: \"kubernetes.io/projected/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-kube-api-access-grbqn\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.418531 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.418515 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-metrics-tls\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.418829 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.418806 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-tmp-dir\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.419042 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.419019 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-config-volume\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.421100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.421083 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-metrics-tls\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.427598 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.427575 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-grbqn\" (UniqueName: \"kubernetes.io/projected/b3aea0eb-2b2f-48f0-b7c0-e4b899544931-kube-api-access-grbqn\") pod \"dns-default-2hg7t\" (UID: \"b3aea0eb-2b2f-48f0-b7c0-e4b899544931\") " pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.462394 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.462327 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:15.593254 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.593206 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbqm9"] Jul 06 22:25:15.595645 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.595622 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-j4mg6"] Jul 06 22:25:15.599534 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:15.599488 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6125683_3f5d_4cc6_ad3d_84b96bae4530.slice/crio-c0f8d44026db418a572b8be5aa7a40ea781f3b0e07d02ac0272d3332e0deb33c WatchSource:0}: Error finding container c0f8d44026db418a572b8be5aa7a40ea781f3b0e07d02ac0272d3332e0deb33c: Status 404 returned error can't find the container with id c0f8d44026db418a572b8be5aa7a40ea781f3b0e07d02ac0272d3332e0deb33c Jul 06 22:25:15.599814 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:15.599792 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc4d630_709f_4f3e_a3ff_4b93fde2bb80.slice/crio-03a7a5685ff7ae5eab646e656987e3ae2d23adc12f7c9dffac80baf36fefe8c6 WatchSource:0}: Error finding container 03a7a5685ff7ae5eab646e656987e3ae2d23adc12f7c9dffac80baf36fefe8c6: Status 404 returned error can't find the container with id 03a7a5685ff7ae5eab646e656987e3ae2d23adc12f7c9dffac80baf36fefe8c6 Jul 06 22:25:15.629986 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.629959 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2hg7t"] Jul 06 22:25:15.634620 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:15.634594 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3aea0eb_2b2f_48f0_b7c0_e4b899544931.slice/crio-77bbfc4b9078b3dfff35e9d14ba5bd5d76753b199fa49efffd0deea57fd95f9c WatchSource:0}: Error finding container 77bbfc4b9078b3dfff35e9d14ba5bd5d76753b199fa49efffd0deea57fd95f9c: Status 404 returned error can't find the container with id 77bbfc4b9078b3dfff35e9d14ba5bd5d76753b199fa49efffd0deea57fd95f9c Jul 06 22:25:15.930290 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.930260 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-dc64cd665-vtzsp"] Jul 06 22:25:15.943485 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.943463 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:15.945366 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.945337 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dc64cd665-vtzsp"] Jul 06 22:25:15.946923 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.946902 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 22:25:15.946923 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.946920 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 22:25:15.947094 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.946954 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 22:25:15.947094 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.947018 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 22:25:15.947094 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.947033 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 22:25:15.947094 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.947058 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dqsw7\"" Jul 06 22:25:15.947339 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.947324 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 22:25:15.947401 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:15.947326 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 22:25:16.023847 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.023819 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-oauth-serving-cert\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.023847 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.023852 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-console-config\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.024076 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.023875 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d98rv\" (UniqueName: \"kubernetes.io/projected/c5134b33-7704-463c-8734-9c6021194d78-kube-api-access-d98rv\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.024076 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.024012 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-serving-cert\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.024076 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.024058 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-service-ca\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.024255 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.024085 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-oauth-config\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.124522 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.124487 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-serving-cert\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.124688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.124534 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-service-ca\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.124688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.124571 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-oauth-config\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.124688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.124622 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-oauth-serving-cert\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.124688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.124651 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-console-config\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.124688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.124675 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d98rv\" (UniqueName: \"kubernetes.io/projected/c5134b33-7704-463c-8734-9c6021194d78-kube-api-access-d98rv\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.126384 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.126331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-service-ca\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.126526 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.126433 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-oauth-serving-cert\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.126526 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.126473 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-console-config\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.128802 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.128611 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-serving-cert\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.128912 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.128850 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-oauth-config\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.134184 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.134160 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d98rv\" (UniqueName: \"kubernetes.io/projected/c5134b33-7704-463c-8734-9c6021194d78-kube-api-access-d98rv\") pod \"console-dc64cd665-vtzsp\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.244152 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.243355 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:16.246321 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.244942 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:16.246713 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.246662 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:25:16.250780 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.249091 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:25:16.250780 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.249162 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-czcdn\"" Jul 06 22:25:16.250780 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.249335 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:25:16.251477 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.251311 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7wdmk\"" Jul 06 22:25:16.253952 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.253630 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:16.440697 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.440622 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dc64cd665-vtzsp"] Jul 06 22:25:16.449014 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:16.448982 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5134b33_7704_463c_8734_9c6021194d78.slice/crio-dbcabfff54d4b62abcc051d7df3c4c4cbfadd85080e8735c1f8ffa0ed1ea0d63 WatchSource:0}: Error finding container dbcabfff54d4b62abcc051d7df3c4c4cbfadd85080e8735c1f8ffa0ed1ea0d63: Status 404 returned error can't find the container with id dbcabfff54d4b62abcc051d7df3c4c4cbfadd85080e8735c1f8ffa0ed1ea0d63 Jul 06 22:25:16.584877 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.584791 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc64cd665-vtzsp" event={"ID":"c5134b33-7704-463c-8734-9c6021194d78","Type":"ContainerStarted","Data":"dbcabfff54d4b62abcc051d7df3c4c4cbfadd85080e8735c1f8ffa0ed1ea0d63"} Jul 06 22:25:16.586805 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.586708 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbqm9" event={"ID":"e6125683-3f5d-4cc6-ad3d-84b96bae4530","Type":"ContainerStarted","Data":"c0f8d44026db418a572b8be5aa7a40ea781f3b0e07d02ac0272d3332e0deb33c"} Jul 06 22:25:16.588607 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.588581 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4mg6" event={"ID":"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80","Type":"ContainerStarted","Data":"03a7a5685ff7ae5eab646e656987e3ae2d23adc12f7c9dffac80baf36fefe8c6"} Jul 06 22:25:16.590299 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.590276 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hg7t" event={"ID":"b3aea0eb-2b2f-48f0-b7c0-e4b899544931","Type":"ContainerStarted","Data":"77bbfc4b9078b3dfff35e9d14ba5bd5d76753b199fa49efffd0deea57fd95f9c"} Jul 06 22:25:16.736630 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.735813 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv"] Jul 06 22:25:16.756927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.755986 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv"] Jul 06 22:25:16.756927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.756100 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:16.760393 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.759412 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-s5xvc\"" Jul 06 22:25:16.760393 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.759651 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 06 22:25:16.830634 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.830479 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11dd52dc-1153-4a2c-9a96-55b262af17ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-46ktv\" (UID: \"11dd52dc-1153-4a2c-9a96-55b262af17ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:16.930966 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:16.930933 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11dd52dc-1153-4a2c-9a96-55b262af17ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-46ktv\" (UID: \"11dd52dc-1153-4a2c-9a96-55b262af17ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:16.931147 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:16.931109 2572 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 06 22:25:16.931208 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:16.931173 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11dd52dc-1153-4a2c-9a96-55b262af17ba-tls-certificates podName:11dd52dc-1153-4a2c-9a96-55b262af17ba nodeName:}" failed. No retries permitted until 2026-07-06 22:25:17.431154428 +0000 UTC m=+49.735323071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/11dd52dc-1153-4a2c-9a96-55b262af17ba-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-46ktv" (UID: "11dd52dc-1153-4a2c-9a96-55b262af17ba") : secret "prometheus-operator-admission-webhook-tls" not found Jul 06 22:25:17.435428 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:17.435378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11dd52dc-1153-4a2c-9a96-55b262af17ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-46ktv\" (UID: \"11dd52dc-1153-4a2c-9a96-55b262af17ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:17.439018 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:17.438985 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/11dd52dc-1153-4a2c-9a96-55b262af17ba-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-46ktv\" (UID: \"11dd52dc-1153-4a2c-9a96-55b262af17ba\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:17.677638 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:17.677601 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:23.054724 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.053425 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv"] Jul 06 22:25:23.387509 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:23.387330 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11dd52dc_1153_4a2c_9a96_55b262af17ba.slice/crio-866b8d17a511000311ca6cfe143f27c73a8e2bda9b67f8851e2a453fd646cefc WatchSource:0}: Error finding container 866b8d17a511000311ca6cfe143f27c73a8e2bda9b67f8851e2a453fd646cefc: Status 404 returned error can't find the container with id 866b8d17a511000311ca6cfe143f27c73a8e2bda9b67f8851e2a453fd646cefc Jul 06 22:25:23.613005 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.612940 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc64cd665-vtzsp" event={"ID":"c5134b33-7704-463c-8734-9c6021194d78","Type":"ContainerStarted","Data":"7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec"} Jul 06 22:25:23.615195 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.615165 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbqm9" event={"ID":"e6125683-3f5d-4cc6-ad3d-84b96bae4530","Type":"ContainerStarted","Data":"125733eaa72b8d394d8f543ce9fe1fbe32ff6d4b6641cdaa0881126eeaa07334"} Jul 06 22:25:23.616562 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.616526 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" event={"ID":"11dd52dc-1153-4a2c-9a96-55b262af17ba","Type":"ContainerStarted","Data":"866b8d17a511000311ca6cfe143f27c73a8e2bda9b67f8851e2a453fd646cefc"} Jul 06 22:25:23.618462 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.618440 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hg7t" event={"ID":"b3aea0eb-2b2f-48f0-b7c0-e4b899544931","Type":"ContainerStarted","Data":"e04fac217de219df7ca1f93ac2a63c494bdcb826384bcf51374f3451e42fd088"} Jul 06 22:25:23.631858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.631816 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-dc64cd665-vtzsp" podStartSLOduration=1.696976023 podStartE2EDuration="8.631799843s" totalImagesPullingTime="6.93482382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:23.630985397 +0000 UTC m=+55.935154056" watchObservedRunningTime="2026-07-06 22:25:23.631799843 +0000 UTC m=+55.935968498" Jul 06 22:25:23.647140 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:23.647095 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbqm9" podStartSLOduration=0.863622493 podStartE2EDuration="8.647079901s" totalImagesPullingTime="7.783457408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:23.646447761 +0000 UTC m=+55.950616419" watchObservedRunningTime="2026-07-06 22:25:23.647079901 +0000 UTC m=+55.951248560" Jul 06 22:25:24.624266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.624216 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4mg6" event={"ID":"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80","Type":"ContainerStarted","Data":"243061f080b1ac2b3311717cd36917ec06f95475e28bd11b78b1f8f434578fdf"} Jul 06 22:25:24.626134 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.626108 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2hg7t" event={"ID":"b3aea0eb-2b2f-48f0-b7c0-e4b899544931","Type":"ContainerStarted","Data":"19ce2aadf6f62e02499fc098097ce4e1400c64198da2090656c2c287826303bd"} Jul 06 22:25:24.644989 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.644939 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2hg7t" podStartSLOduration=1.896259387 podStartE2EDuration="9.64491876s" totalImagesPullingTime="7.748659373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:24.64410542 +0000 UTC m=+56.948274080" watchObservedRunningTime="2026-07-06 22:25:24.64491876 +0000 UTC m=+56.949087424" Jul 06 22:25:24.685730 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.685695 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5665b66699-ph4z8"] Jul 06 22:25:24.687469 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.687450 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.698443 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.698418 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 22:25:24.702760 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.702733 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5665b66699-ph4z8"] Jul 06 22:25:24.789322 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789288 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-oauth-config\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.789322 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789326 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-service-ca\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.789532 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789346 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-serving-cert\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.789532 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789404 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gztqj\" (UniqueName: \"kubernetes.io/projected/f902de25-dcb2-4093-b95a-216256d78d98-kube-api-access-gztqj\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.789532 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-console-config\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.789532 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789488 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-trusted-ca-bundle\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.789532 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.789518 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-oauth-serving-cert\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.890837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.890757 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-oauth-config\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.890837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.890795 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-service-ca\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.890837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.890812 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-serving-cert\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891052 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.890961 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gztqj\" (UniqueName: \"kubernetes.io/projected/f902de25-dcb2-4093-b95a-216256d78d98-kube-api-access-gztqj\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891052 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.891045 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-console-config\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891134 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.891079 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-trusted-ca-bundle\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891134 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.891110 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-oauth-serving-cert\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891855 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.891711 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-service-ca\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891855 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.891779 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-oauth-serving-cert\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.891855 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.891800 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-console-config\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.892068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.892052 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-trusted-ca-bundle\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.894714 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.894692 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-oauth-config\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.894804 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.894765 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-serving-cert\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:24.900716 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:24.900697 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gztqj\" (UniqueName: \"kubernetes.io/projected/f902de25-dcb2-4093-b95a-216256d78d98-kube-api-access-gztqj\") pod \"console-5665b66699-ph4z8\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:25.000337 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.000280 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:25.138141 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.138109 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5665b66699-ph4z8"] Jul 06 22:25:25.141187 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:25.141122 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf902de25_dcb2_4093_b95a_216256d78d98.slice/crio-35d8f0a184edca3fb2d4c76b600fb69b38dab9336f955b1915183ab74fc9bbbd WatchSource:0}: Error finding container 35d8f0a184edca3fb2d4c76b600fb69b38dab9336f955b1915183ab74fc9bbbd: Status 404 returned error can't find the container with id 35d8f0a184edca3fb2d4c76b600fb69b38dab9336f955b1915183ab74fc9bbbd Jul 06 22:25:25.631620 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.631573 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b66699-ph4z8" event={"ID":"f902de25-dcb2-4093-b95a-216256d78d98","Type":"ContainerStarted","Data":"f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f"} Jul 06 22:25:25.632033 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.631631 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b66699-ph4z8" event={"ID":"f902de25-dcb2-4093-b95a-216256d78d98","Type":"ContainerStarted","Data":"35d8f0a184edca3fb2d4c76b600fb69b38dab9336f955b1915183ab74fc9bbbd"} Jul 06 22:25:25.633710 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.633682 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4mg6" event={"ID":"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80","Type":"ContainerStarted","Data":"6cc7e5d322f57a9059c568589375ffc4d618400cf97f79f92abbde4d36c20040"} Jul 06 22:25:25.635364 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.635336 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" event={"ID":"11dd52dc-1153-4a2c-9a96-55b262af17ba","Type":"ContainerStarted","Data":"f79c4047661a2908c84b8d194cb96ec472b969846ded8a96ebf38aa2764aed3c"} Jul 06 22:25:25.635497 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.635479 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:25.650039 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.649986 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5665b66699-ph4z8" podStartSLOduration=1.649972205 podStartE2EDuration="1.649972205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:25.648700664 +0000 UTC m=+57.952869323" watchObservedRunningTime="2026-07-06 22:25:25.649972205 +0000 UTC m=+57.954140860" Jul 06 22:25:25.664118 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:25.664064 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" podStartSLOduration=8.463358855 podStartE2EDuration="9.66404958s" totalImagesPullingTime="1.200690725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:25.663187293 +0000 UTC m=+57.967355958" watchObservedRunningTime="2026-07-06 22:25:25.66404958 +0000 UTC m=+57.968218239" Jul 06 22:25:26.254285 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.254252 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:26.254474 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.254320 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:25:26.255916 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.255878 2572 patch_prober.go:28] interesting pod/console-dc64cd665-vtzsp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body= Jul 06 22:25:26.256055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.255952 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-dc64cd665-vtzsp" podUID="c5134b33-7704-463c-8734-9c6021194d78" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" Jul 06 22:25:26.640462 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.640391 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:26.646586 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.646561 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-46ktv" Jul 06 22:25:26.813816 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.813785 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll"] Jul 06 22:25:26.822965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.822939 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.825925 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.826187 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.826511 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mtg7t\"" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.826717 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.827083 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.827094 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 22:25:26.827200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.827130 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll"] Jul 06 22:25:26.908043 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.907963 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80072443-7f2e-4c71-b73c-6f63230582ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:26.908194 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.908058 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmvrx\" (UniqueName: \"kubernetes.io/projected/80072443-7f2e-4c71-b73c-6f63230582ae-kube-api-access-xmvrx\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:26.908194 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.908104 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80072443-7f2e-4c71-b73c-6f63230582ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:26.908194 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:26.908123 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80072443-7f2e-4c71-b73c-6f63230582ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.009270 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.008764 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80072443-7f2e-4c71-b73c-6f63230582ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.009270 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.008816 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80072443-7f2e-4c71-b73c-6f63230582ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.009270 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.008847 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80072443-7f2e-4c71-b73c-6f63230582ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.009270 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.008920 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmvrx\" (UniqueName: \"kubernetes.io/projected/80072443-7f2e-4c71-b73c-6f63230582ae-kube-api-access-xmvrx\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.009600 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.009559 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/80072443-7f2e-4c71-b73c-6f63230582ae-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.012048 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.012016 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/80072443-7f2e-4c71-b73c-6f63230582ae-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.012201 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.012177 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/80072443-7f2e-4c71-b73c-6f63230582ae-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.020848 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.020818 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmvrx\" (UniqueName: \"kubernetes.io/projected/80072443-7f2e-4c71-b73c-6f63230582ae-kube-api-access-xmvrx\") pod \"prometheus-operator-567f7b8b7c-5fnll\" (UID: \"80072443-7f2e-4c71-b73c-6f63230582ae\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.135020 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.134996 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" Jul 06 22:25:27.263050 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.263002 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll"] Jul 06 22:25:27.265616 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:27.265591 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80072443_7f2e_4c71_b73c_6f63230582ae.slice/crio-f9e8ddb05bc597c34401da03aaf880e2b39dc9edb74a43e1a82834f8c06b4ca1 WatchSource:0}: Error finding container f9e8ddb05bc597c34401da03aaf880e2b39dc9edb74a43e1a82834f8c06b4ca1: Status 404 returned error can't find the container with id f9e8ddb05bc597c34401da03aaf880e2b39dc9edb74a43e1a82834f8c06b4ca1 Jul 06 22:25:27.644878 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.644783 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-j4mg6" event={"ID":"7fc4d630-709f-4f3e-a3ff-4b93fde2bb80","Type":"ContainerStarted","Data":"314d136385091705a254a45f7cadd1d8d9c935d2572b8733ca107b5a5455b3a3"} Jul 06 22:25:27.646006 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.645980 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" event={"ID":"80072443-7f2e-4c71-b73c-6f63230582ae","Type":"ContainerStarted","Data":"f9e8ddb05bc597c34401da03aaf880e2b39dc9edb74a43e1a82834f8c06b4ca1"} Jul 06 22:25:27.663095 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:27.663047 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-j4mg6" podStartSLOduration=1.302860256 podStartE2EDuration="12.663034703s" totalImagesPullingTime="11.360174447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:25:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:27.662037896 +0000 UTC m=+59.966206566" watchObservedRunningTime="2026-07-06 22:25:27.663034703 +0000 UTC m=+59.967203380" Jul 06 22:25:28.651271 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:28.651159 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" event={"ID":"80072443-7f2e-4c71-b73c-6f63230582ae","Type":"ContainerStarted","Data":"c14f45d93d4fdacf4549beb31ce1bfb39440b36a058a448e336aa6501f52dff6"} Jul 06 22:25:28.651271 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:28.651223 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" event={"ID":"80072443-7f2e-4c71-b73c-6f63230582ae","Type":"ContainerStarted","Data":"24149e76337554e0c8aaba96f7857cfdf7c6f293c177d4ea7e841de9865a922f"} Jul 06 22:25:28.673014 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:28.672967 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-5fnll" podStartSLOduration=1.569480709 podStartE2EDuration="2.672951796s" totalImagesPullingTime="1.103471087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:28.670776802 +0000 UTC m=+60.974945462" watchObservedRunningTime="2026-07-06 22:25:28.672951796 +0000 UTC m=+60.977120455" Jul 06 22:25:29.559074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:29.559048 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7tdp7" Jul 06 22:25:30.138362 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.138324 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp"] Jul 06 22:25:30.165683 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.165651 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp"] Jul 06 22:25:30.165824 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.165783 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.168965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.168942 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-mnllk\"" Jul 06 22:25:30.169277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.169258 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 06 22:25:30.169400 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.169339 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 06 22:25:30.195510 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.195475 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc"] Jul 06 22:25:30.211993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.211962 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc"] Jul 06 22:25:30.212167 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.212101 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.215061 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.215038 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 06 22:25:30.215372 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.215351 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 06 22:25:30.215651 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.215632 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-8896d\"" Jul 06 22:25:30.216290 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.216269 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 06 22:25:30.234064 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.234035 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8g4nd\" (UniqueName: \"kubernetes.io/projected/a0cb8137-ca52-442f-b047-6fc455e12f2f-kube-api-access-8g4nd\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.234193 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.234091 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0cb8137-ca52-442f-b047-6fc455e12f2f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.234193 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.234111 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0cb8137-ca52-442f-b047-6fc455e12f2f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.234193 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.234133 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0cb8137-ca52-442f-b047-6fc455e12f2f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.240528 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.240504 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-79sg4"] Jul 06 22:25:30.261163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.261124 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.273180 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.273145 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 22:25:30.273361 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.273298 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-svfcn\"" Jul 06 22:25:30.275944 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.275925 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 22:25:30.282972 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.282943 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 22:25:30.335094 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335064 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84zk8\" (UniqueName: \"kubernetes.io/projected/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-api-access-84zk8\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.335286 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335105 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0cb8137-ca52-442f-b047-6fc455e12f2f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.335286 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335126 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0cb8137-ca52-442f-b047-6fc455e12f2f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.335286 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335147 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.335286 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335172 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-tls\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335286 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335199 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0cb8137-ca52-442f-b047-6fc455e12f2f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.335546 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335318 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9454bdb1-f37d-4995-a1fc-0a668f8b351b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.335546 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-root\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335546 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335454 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-metrics-client-ca\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335546 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9454bdb1-f37d-4995-a1fc-0a668f8b351b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.335735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335554 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335620 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfzdm\" (UniqueName: \"kubernetes.io/projected/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-kube-api-access-gfzdm\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335662 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.335735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335693 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8g4nd\" (UniqueName: \"kubernetes.io/projected/a0cb8137-ca52-442f-b047-6fc455e12f2f-kube-api-access-8g4nd\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.335735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335717 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-textfile\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335979 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335738 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-wtmp\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335979 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335771 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.335979 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335828 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-sys\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335979 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335851 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-accelerators-collector-config\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.335979 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.335966 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0cb8137-ca52-442f-b047-6fc455e12f2f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.337973 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.337936 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0cb8137-ca52-442f-b047-6fc455e12f2f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.338559 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.338533 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a0cb8137-ca52-442f-b047-6fc455e12f2f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.357392 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.357356 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8g4nd\" (UniqueName: \"kubernetes.io/projected/a0cb8137-ca52-442f-b047-6fc455e12f2f-kube-api-access-8g4nd\") pod \"openshift-state-metrics-54cf96dfcf-zl6bp\" (UID: \"a0cb8137-ca52-442f-b047-6fc455e12f2f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.437082 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437051 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-metrics-client-ca\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437272 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437107 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9454bdb1-f37d-4995-a1fc-0a668f8b351b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437272 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437227 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437354 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437332 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfzdm\" (UniqueName: \"kubernetes.io/projected/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-kube-api-access-gfzdm\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437392 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437356 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437392 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437378 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-textfile\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437467 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-wtmp\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437467 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437429 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437544 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437472 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-sys\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437544 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-accelerators-collector-config\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437544 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437525 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84zk8\" (UniqueName: \"kubernetes.io/projected/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-api-access-84zk8\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437544 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437531 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/9454bdb1-f37d-4995-a1fc-0a668f8b351b-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437719 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437564 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437719 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437592 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-tls\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437719 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437616 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9454bdb1-f37d-4995-a1fc-0a668f8b351b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.437719 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437637 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-root\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437914 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437719 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-root\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437914 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437732 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-metrics-client-ca\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.437914 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.437758 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-sys\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.438252 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.438210 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-wtmp\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.438401 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.438376 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-textfile\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.438597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.438574 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9454bdb1-f37d-4995-a1fc-0a668f8b351b-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.438597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.438583 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.438757 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.438661 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-accelerators-collector-config\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.440069 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.440049 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.440275 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.440253 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.440413 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.440397 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.440520 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.440503 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-node-exporter-tls\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.447837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.447812 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfzdm\" (UniqueName: \"kubernetes.io/projected/5f2dfeb0-402f-46a7-8c00-71b2edad6bbb-kube-api-access-gfzdm\") pod \"node-exporter-79sg4\" (UID: \"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb\") " pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.448286 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.448267 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84zk8\" (UniqueName: \"kubernetes.io/projected/9454bdb1-f37d-4995-a1fc-0a668f8b351b-kube-api-access-84zk8\") pod \"kube-state-metrics-7c9dbb8b7d-fpsmc\" (UID: \"9454bdb1-f37d-4995-a1fc-0a668f8b351b\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.474677 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.474646 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" Jul 06 22:25:30.522178 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.522150 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" Jul 06 22:25:30.574745 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.574718 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-79sg4" Jul 06 22:25:30.611157 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:30.611108 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f2dfeb0_402f_46a7_8c00_71b2edad6bbb.slice/crio-89ae9dad69bd4db5df31934307ebe299696a43f202be263c66f078c3acdf286d WatchSource:0}: Error finding container 89ae9dad69bd4db5df31934307ebe299696a43f202be263c66f078c3acdf286d: Status 404 returned error can't find the container with id 89ae9dad69bd4db5df31934307ebe299696a43f202be263c66f078c3acdf286d Jul 06 22:25:30.653793 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.653742 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp"] Jul 06 22:25:30.657219 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:30.657192 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0cb8137_ca52_442f_b047_6fc455e12f2f.slice/crio-4717694c7400c302aa2c17d9aab62a01cc76ec850428e43fa62ea088ca0c5c46 WatchSource:0}: Error finding container 4717694c7400c302aa2c17d9aab62a01cc76ec850428e43fa62ea088ca0c5c46: Status 404 returned error can't find the container with id 4717694c7400c302aa2c17d9aab62a01cc76ec850428e43fa62ea088ca0c5c46 Jul 06 22:25:30.660142 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.660039 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79sg4" event={"ID":"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb","Type":"ContainerStarted","Data":"89ae9dad69bd4db5df31934307ebe299696a43f202be263c66f078c3acdf286d"} Jul 06 22:25:30.695217 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:30.695192 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc"] Jul 06 22:25:30.711359 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:30.711329 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9454bdb1_f37d_4995_a1fc_0a668f8b351b.slice/crio-a9439354c334610a4ac9996c7c7b137ecc36a86bfb6c7be83f9460e17f34fb48 WatchSource:0}: Error finding container a9439354c334610a4ac9996c7c7b137ecc36a86bfb6c7be83f9460e17f34fb48: Status 404 returned error can't find the container with id a9439354c334610a4ac9996c7c7b137ecc36a86bfb6c7be83f9460e17f34fb48 Jul 06 22:25:31.182435 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.182396 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 22:25:31.185567 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.185541 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.189318 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.189290 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 06 22:25:31.189318 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.189306 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 06 22:25:31.189511 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.189441 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 06 22:25:31.189789 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.189623 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 06 22:25:31.189789 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.189727 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zgnsk\"" Jul 06 22:25:31.190605 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.190052 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 06 22:25:31.190605 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.190183 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 06 22:25:31.190605 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.190054 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 06 22:25:31.190605 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.190453 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 06 22:25:31.195279 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.194724 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 06 22:25:31.231296 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.231171 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 22:25:31.245561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245528 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-web-config\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245582 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245620 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245656 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-config-out\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245729 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-config-volume\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245779 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245798 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245817 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245832 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9zdr\" (UniqueName: \"kubernetes.io/projected/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-kube-api-access-z9zdr\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245873 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.245983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.245901 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346375 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346344 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-web-config\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346418 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346445 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-config-out\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346466 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346495 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-config-volume\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346549 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346549 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346567 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346582 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346597 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9zdr\" (UniqueName: \"kubernetes.io/projected/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-kube-api-access-z9zdr\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346621 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.346960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.346653 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.347821 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.347794 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.347962 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:31.347945 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-trusted-ca-bundle podName:3b2d387e-c01a-4b0f-8eea-6425ed5e8d42 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:31.847923684 +0000 UTC m=+64.152092337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "3b2d387e-c01a-4b0f-8eea-6425ed5e8d42") : configmap references non-existent config key: ca-bundle.crt Jul 06 22:25:31.349204 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.349175 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.349621 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:31.349600 2572 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 06 22:25:31.349709 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:25:31.349673 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-main-tls podName:3b2d387e-c01a-4b0f-8eea-6425ed5e8d42 nodeName:}" failed. No retries permitted until 2026-07-06 22:25:31.849653628 +0000 UTC m=+64.153822284 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3b2d387e-c01a-4b0f-8eea-6425ed5e8d42") : secret "alertmanager-main-tls" not found Jul 06 22:25:31.352581 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.352555 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-config-out\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.353030 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.353008 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.353284 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.353261 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.353432 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.353388 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-config-volume\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.353432 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.353397 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.354705 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.354687 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-web-config\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.355174 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.355135 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.356414 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.356370 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.362120 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.362093 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9zdr\" (UniqueName: \"kubernetes.io/projected/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-kube-api-access-z9zdr\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.665275 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.665172 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" event={"ID":"9454bdb1-f37d-4995-a1fc-0a668f8b351b","Type":"ContainerStarted","Data":"a9439354c334610a4ac9996c7c7b137ecc36a86bfb6c7be83f9460e17f34fb48"} Jul 06 22:25:31.667031 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.667008 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" event={"ID":"a0cb8137-ca52-442f-b047-6fc455e12f2f","Type":"ContainerStarted","Data":"4937232a27431eacff67d86a5fd19b3128cecf4555347b15f113682c1b3789c5"} Jul 06 22:25:31.667134 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.667052 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" event={"ID":"a0cb8137-ca52-442f-b047-6fc455e12f2f","Type":"ContainerStarted","Data":"16cea3ff2ce5bd58daffbaf0eac187911e9db3648934af4555cd90f4e2226e50"} Jul 06 22:25:31.667134 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.667069 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" event={"ID":"a0cb8137-ca52-442f-b047-6fc455e12f2f","Type":"ContainerStarted","Data":"4717694c7400c302aa2c17d9aab62a01cc76ec850428e43fa62ea088ca0c5c46"} Jul 06 22:25:31.851042 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.851003 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.851215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.851114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.854133 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.854014 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:31.858255 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:31.858204 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3b2d387e-c01a-4b0f-8eea-6425ed5e8d42-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42\") " pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:32.099032 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.098987 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 06 22:25:32.403399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.403181 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 06 22:25:32.410886 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:32.410848 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b2d387e_c01a_4b0f_8eea_6425ed5e8d42.slice/crio-0a6c1538ba70cd8eb392d681b3b13f1052ce66a1cc0666b0733fa8e8b186efb9 WatchSource:0}: Error finding container 0a6c1538ba70cd8eb392d681b3b13f1052ce66a1cc0666b0733fa8e8b186efb9: Status 404 returned error can't find the container with id 0a6c1538ba70cd8eb392d681b3b13f1052ce66a1cc0666b0733fa8e8b186efb9 Jul 06 22:25:32.672381 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.672168 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"0a6c1538ba70cd8eb392d681b3b13f1052ce66a1cc0666b0733fa8e8b186efb9"} Jul 06 22:25:32.675105 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.675066 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" event={"ID":"9454bdb1-f37d-4995-a1fc-0a668f8b351b","Type":"ContainerStarted","Data":"8d56280fc3b03f148fb7b0409de398ccaeaf332de7bc8d5e0a996eb5b50cae13"} Jul 06 22:25:32.675275 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.675138 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" event={"ID":"9454bdb1-f37d-4995-a1fc-0a668f8b351b","Type":"ContainerStarted","Data":"62399a2bd92d5e04cec4aef99e15706034db5aee131d78db2e6bda31889b8758"} Jul 06 22:25:32.675275 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.675164 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" event={"ID":"9454bdb1-f37d-4995-a1fc-0a668f8b351b","Type":"ContainerStarted","Data":"4ae441699e95fc01dc68146ba948f9b4ce5fd1feaa382b4d446e00e20f5feb4d"} Jul 06 22:25:32.677767 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.677741 2572 generic.go:356] "Generic (PLEG): container finished" podID="5f2dfeb0-402f-46a7-8c00-71b2edad6bbb" containerID="61a0905b7aadd5e2c3a56f78044833acdb3697d2b20eb48a42230f193d2c7864" exitCode=0 Jul 06 22:25:32.677895 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.677810 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79sg4" event={"ID":"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb","Type":"ContainerDied","Data":"61a0905b7aadd5e2c3a56f78044833acdb3697d2b20eb48a42230f193d2c7864"} Jul 06 22:25:32.680264 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.680220 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" event={"ID":"a0cb8137-ca52-442f-b047-6fc455e12f2f","Type":"ContainerStarted","Data":"b80772169d56a3d7a10c0d368c3fb214647792ea0d2834227b99944e555985ee"} Jul 06 22:25:32.697280 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.697198 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fpsmc" podStartSLOduration=1.19657204 podStartE2EDuration="2.697174693s" totalImagesPullingTime="1.500602653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:32.696629057 +0000 UTC m=+65.000797715" watchObservedRunningTime="2026-07-06 22:25:32.697174693 +0000 UTC m=+65.001343355" Jul 06 22:25:32.724314 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.724262 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-zl6bp" podStartSLOduration=1.337980395 podStartE2EDuration="2.724221948s" totalImagesPullingTime="1.386241553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:32.72225466 +0000 UTC m=+65.026423355" watchObservedRunningTime="2026-07-06 22:25:32.724221948 +0000 UTC m=+65.028390607" Jul 06 22:25:32.861762 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.861727 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:32.864723 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.864692 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:25:32.875076 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.875038 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/158c4ff9-36f2-454d-8082-94ab9b2e5bdb-metrics-certs\") pod \"network-metrics-daemon-2sj22\" (UID: \"158c4ff9-36f2-454d-8082-94ab9b2e5bdb\") " pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:32.963041 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.963001 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:32.966295 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.966269 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:25:32.975933 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:32.975912 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8b4cf0a7-678c-4f21-8d45-a46b9e1dd522-original-pull-secret\") pod \"global-pull-secret-syncer-vlvg2\" (UID: \"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522\") " pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:33.063962 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.063926 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:33.064123 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.063987 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7wdmk\"" Jul 06 22:25:33.066687 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.066660 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:25:33.072568 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.072301 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2sj22" Jul 06 22:25:33.076841 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.076809 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:25:33.087472 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.087446 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrvjf\" (UniqueName: \"kubernetes.io/projected/361ac06c-1abe-49fd-bd1a-4f8083a9e962-kube-api-access-jrvjf\") pod \"network-check-target-xrmgf\" (UID: \"361ac06c-1abe-49fd-bd1a-4f8083a9e962\") " pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:33.217055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.215764 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2sj22"] Jul 06 22:25:33.221734 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.221703 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7c684f54f5-bssqq"] Jul 06 22:25:33.225423 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.225400 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.228405 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228382 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 22:25:33.228519 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228472 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 22:25:33.228584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228529 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-f89sv\"" Jul 06 22:25:33.228992 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228761 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 22:25:33.228992 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228763 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 22:25:33.228992 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228764 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-4g5tetmtedhp3\"" Jul 06 22:25:33.228992 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.228803 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 22:25:33.238146 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.238120 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c684f54f5-bssqq"] Jul 06 22:25:33.250629 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.250600 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-vlvg2" Jul 06 22:25:33.265206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265172 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265362 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265215 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265362 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265271 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wqmz\" (UniqueName: \"kubernetes.io/projected/d77792fd-44c9-4ebb-ab23-0e690f960164-kube-api-access-8wqmz\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265362 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265331 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265522 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d77792fd-44c9-4ebb-ab23-0e690f960164-metrics-client-ca\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265522 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265386 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-grpc-tls\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265522 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265406 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-tls\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.265639 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.265516 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366155 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366118 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366171 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d77792fd-44c9-4ebb-ab23-0e690f960164-metrics-client-ca\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366195 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-grpc-tls\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-tls\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366263 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366298 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366333 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.366371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.366358 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wqmz\" (UniqueName: \"kubernetes.io/projected/d77792fd-44c9-4ebb-ab23-0e690f960164-kube-api-access-8wqmz\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.367704 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.367643 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d77792fd-44c9-4ebb-ab23-0e690f960164-metrics-client-ca\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.370264 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.369898 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-tls\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.370264 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.369981 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.371300 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.370793 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.371419 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.371397 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.371574 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.371553 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-grpc-tls\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.371965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.371944 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d77792fd-44c9-4ebb-ab23-0e690f960164-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.372764 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.372740 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-czcdn\"" Jul 06 22:25:33.375160 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.375138 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wqmz\" (UniqueName: \"kubernetes.io/projected/d77792fd-44c9-4ebb-ab23-0e690f960164-kube-api-access-8wqmz\") pod \"thanos-querier-7c684f54f5-bssqq\" (UID: \"d77792fd-44c9-4ebb-ab23-0e690f960164\") " pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.380588 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.380563 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:33.521299 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.521271 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-vlvg2"] Jul 06 22:25:33.523001 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:33.522973 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4cf0a7_678c_4f21_8d45_a46b9e1dd522.slice/crio-84eb49ff1788c248de82d3cc5ed8e7462e4a944a950500df84c2b9af5302edef WatchSource:0}: Error finding container 84eb49ff1788c248de82d3cc5ed8e7462e4a944a950500df84c2b9af5302edef: Status 404 returned error can't find the container with id 84eb49ff1788c248de82d3cc5ed8e7462e4a944a950500df84c2b9af5302edef Jul 06 22:25:33.536365 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.536337 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:33.558407 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.558374 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xrmgf"] Jul 06 22:25:33.561181 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:33.561156 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361ac06c_1abe_49fd_bd1a_4f8083a9e962.slice/crio-fb01347d9da0299a24ea12ba897bc9495e9e357c9788e9adcab0f332bd380110 WatchSource:0}: Error finding container fb01347d9da0299a24ea12ba897bc9495e9e357c9788e9adcab0f332bd380110: Status 404 returned error can't find the container with id fb01347d9da0299a24ea12ba897bc9495e9e357c9788e9adcab0f332bd380110 Jul 06 22:25:33.678901 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.678866 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7c684f54f5-bssqq"] Jul 06 22:25:33.682243 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:33.682197 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd77792fd_44c9_4ebb_ab23_0e690f960164.slice/crio-9564093967f114b21a91002be8fa69efdefcaa1743c235f6ddf6dbc453f497d7 WatchSource:0}: Error finding container 9564093967f114b21a91002be8fa69efdefcaa1743c235f6ddf6dbc453f497d7: Status 404 returned error can't find the container with id 9564093967f114b21a91002be8fa69efdefcaa1743c235f6ddf6dbc453f497d7 Jul 06 22:25:33.685532 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.685503 2572 generic.go:356] "Generic (PLEG): container finished" podID="3b2d387e-c01a-4b0f-8eea-6425ed5e8d42" containerID="c9bd20f3bb59ea0e4feaf60161bea6950eff0cd892b706be8eb9926c46415a1e" exitCode=0 Jul 06 22:25:33.685655 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.685604 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerDied","Data":"c9bd20f3bb59ea0e4feaf60161bea6950eff0cd892b706be8eb9926c46415a1e"} Jul 06 22:25:33.688154 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.688128 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2sj22" event={"ID":"158c4ff9-36f2-454d-8082-94ab9b2e5bdb","Type":"ContainerStarted","Data":"c48370737817f381396698d9f861b94bdcfe2323902c1ad5a540c8c28e164e91"} Jul 06 22:25:33.689899 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.689875 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xrmgf" event={"ID":"361ac06c-1abe-49fd-bd1a-4f8083a9e962","Type":"ContainerStarted","Data":"fb01347d9da0299a24ea12ba897bc9495e9e357c9788e9adcab0f332bd380110"} Jul 06 22:25:33.692043 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.692019 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79sg4" event={"ID":"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb","Type":"ContainerStarted","Data":"65f1a236b0ac024b96aeff6f80d7bbe920ddf9974e3fb8f0ddace6a2d716a007"} Jul 06 22:25:33.692139 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.692067 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-79sg4" event={"ID":"5f2dfeb0-402f-46a7-8c00-71b2edad6bbb","Type":"ContainerStarted","Data":"f076717d47eda8702404396f55f8fb23ee597b0743b578ca0215e8b4d7abf02d"} Jul 06 22:25:33.693211 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.693193 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlvg2" event={"ID":"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522","Type":"ContainerStarted","Data":"84eb49ff1788c248de82d3cc5ed8e7462e4a944a950500df84c2b9af5302edef"} Jul 06 22:25:33.737811 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:33.737711 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-79sg4" podStartSLOduration=1.9895994780000001 podStartE2EDuration="3.737695152s" totalImagesPullingTime="1.599031255s" totalInitContainerRuntime="149.064419ms" isStatefulPod=true podCreationTimestamp="2026-07-06 22:25:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:33.737158615 +0000 UTC m=+66.041327271" watchObservedRunningTime="2026-07-06 22:25:33.737695152 +0000 UTC m=+66.041863811" Jul 06 22:25:34.630452 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.629473 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf"] Jul 06 22:25:34.635096 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.635059 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.638693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.638393 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 06 22:25:34.638693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.638624 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 06 22:25:34.640106 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.639734 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5g2r5\"" Jul 06 22:25:34.640106 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.639800 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 06 22:25:34.640106 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.639934 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-b1q35i4sajcif\"" Jul 06 22:25:34.640106 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.640022 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 06 22:25:34.645101 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.644372 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf"] Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.678775 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3d957cc-6cf8-4f15-a836-1f8392b79312-audit-log\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.678829 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-secret-metrics-server-tls\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.678897 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mqz8\" (UniqueName: \"kubernetes.io/projected/d3d957cc-6cf8-4f15-a836-1f8392b79312-kube-api-access-9mqz8\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.678926 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-secret-metrics-server-client-certs\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.678970 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3d957cc-6cf8-4f15-a836-1f8392b79312-metrics-server-audit-profiles\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.679004 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-client-ca-bundle\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.679164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.679043 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d957cc-6cf8-4f15-a836-1f8392b79312-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.703290 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.703261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"9564093967f114b21a91002be8fa69efdefcaa1743c235f6ddf6dbc453f497d7"} Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.780780 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-secret-metrics-server-tls\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.780914 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mqz8\" (UniqueName: \"kubernetes.io/projected/d3d957cc-6cf8-4f15-a836-1f8392b79312-kube-api-access-9mqz8\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.780950 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-secret-metrics-server-client-certs\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.781007 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3d957cc-6cf8-4f15-a836-1f8392b79312-metrics-server-audit-profiles\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.781043 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-client-ca-bundle\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.781082 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d957cc-6cf8-4f15-a836-1f8392b79312-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.781133 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3d957cc-6cf8-4f15-a836-1f8392b79312-audit-log\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.782215 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.781548 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d3d957cc-6cf8-4f15-a836-1f8392b79312-audit-log\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.784180 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.784136 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d3d957cc-6cf8-4f15-a836-1f8392b79312-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.784759 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.784591 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d3d957cc-6cf8-4f15-a836-1f8392b79312-metrics-server-audit-profiles\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.784759 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.784723 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-secret-metrics-server-tls\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.786724 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.786695 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-secret-metrics-server-client-certs\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.787592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.787567 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d3d957cc-6cf8-4f15-a836-1f8392b79312-client-ca-bundle\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.797280 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.797171 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mqz8\" (UniqueName: \"kubernetes.io/projected/d3d957cc-6cf8-4f15-a836-1f8392b79312-kube-api-access-9mqz8\") pod \"metrics-server-5c86cd7fb9-l6rpf\" (UID: \"d3d957cc-6cf8-4f15-a836-1f8392b79312\") " pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.956896 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.956859 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:34.958861 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.958832 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dc64cd665-vtzsp"] Jul 06 22:25:34.984319 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.984247 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cc97cb67-qp87k"] Jul 06 22:25:34.991004 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:34.990924 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.003792 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.000648 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:35.003792 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.000705 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc97cb67-qp87k"] Jul 06 22:25:35.003792 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.001677 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:25:35.003792 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.002116 2572 patch_prober.go:28] interesting pod/console-5665b66699-ph4z8 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body= Jul 06 22:25:35.003792 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.002160 2572 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5665b66699-ph4z8" podUID="f902de25-dcb2-4093-b95a-216256d78d98" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.086842 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-serving-cert\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.086904 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-service-ca\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.086924 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-oauth-serving-cert\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.087006 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcdf\" (UniqueName: \"kubernetes.io/projected/704d460e-28a7-418f-88e8-582535c76815-kube-api-access-dgcdf\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.087039 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-trusted-ca-bundle\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.087073 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-console-config\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.088025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.087100 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-oauth-config\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.149193 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.147813 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf"] Jul 06 22:25:35.188801 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.188510 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-console-config\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.188801 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.188556 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-oauth-config\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.188801 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.188590 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-serving-cert\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.188801 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.188631 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-service-ca\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.190521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.189819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-oauth-serving-cert\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.190521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.189879 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-oauth-serving-cert\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.190521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.189907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcdf\" (UniqueName: \"kubernetes.io/projected/704d460e-28a7-418f-88e8-582535c76815-kube-api-access-dgcdf\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.190521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.189953 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-trusted-ca-bundle\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.190837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.190540 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-console-config\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.191611 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.191525 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-service-ca\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.191611 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.191510 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-trusted-ca-bundle\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.192634 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.192555 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-oauth-config\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.193769 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.193747 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-serving-cert\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.199708 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.199660 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcdf\" (UniqueName: \"kubernetes.io/projected/704d460e-28a7-418f-88e8-582535c76815-kube-api-access-dgcdf\") pod \"console-7cc97cb67-qp87k\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.305999 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.305968 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:35.495733 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.494030 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc"] Jul 06 22:25:35.498112 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.498090 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.507566 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.504359 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 06 22:25:35.507566 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.504646 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-btwj8\"" Jul 06 22:25:35.507566 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.504868 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 06 22:25:35.507566 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.505135 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 06 22:25:35.507566 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.505499 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 06 22:25:35.507566 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.505577 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 06 22:25:35.511099 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.511074 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 06 22:25:35.517928 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.517737 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc"] Jul 06 22:25:35.597430 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597389 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-telemeter-client-tls\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597576 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597464 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-metrics-client-ca\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597576 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597493 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597576 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597528 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-secret-telemeter-client\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597676 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597575 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x47c8\" (UniqueName: \"kubernetes.io/projected/188689f4-7e55-4876-87bc-0bc264dc6205-kube-api-access-x47c8\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597676 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597643 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-federate-client-tls\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597774 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597682 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-serving-certs-ca-bundle\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.597774 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.597722 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.642181 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.642148 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2hg7t" Jul 06 22:25:35.698537 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698500 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-metrics-client-ca\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698725 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698555 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698725 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698586 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-secret-telemeter-client\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698725 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698631 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x47c8\" (UniqueName: \"kubernetes.io/projected/188689f4-7e55-4876-87bc-0bc264dc6205-kube-api-access-x47c8\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698725 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698701 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-federate-client-tls\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698942 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-serving-certs-ca-bundle\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698942 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698806 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.698942 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.698859 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-telemeter-client-tls\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.699401 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.699152 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-metrics-client-ca\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.700739 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.699961 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-serving-certs-ca-bundle\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.700739 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.700685 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/188689f4-7e55-4876-87bc-0bc264dc6205-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.703975 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.703944 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.709161 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.709138 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-telemeter-client-tls\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.709630 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.709576 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-secret-telemeter-client\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.715625 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.715598 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/188689f4-7e55-4876-87bc-0bc264dc6205-federate-client-tls\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.716226 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.716194 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" event={"ID":"d3d957cc-6cf8-4f15-a836-1f8392b79312","Type":"ContainerStarted","Data":"cfbf3a5d68feeedf440d6e6aef69b919eab3ccccf33e965a5d8c22a2969c8bc6"} Jul 06 22:25:35.719157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.719132 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x47c8\" (UniqueName: \"kubernetes.io/projected/188689f4-7e55-4876-87bc-0bc264dc6205-kube-api-access-x47c8\") pod \"telemeter-client-6b6c44fc4d-fvlwc\" (UID: \"188689f4-7e55-4876-87bc-0bc264dc6205\") " pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:35.720800 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.720769 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2sj22" event={"ID":"158c4ff9-36f2-454d-8082-94ab9b2e5bdb","Type":"ContainerStarted","Data":"d48fda0a7daf4b0c842ba8a29d28d8ded9e544c3e0c7179ce849335a8df4ca17"} Jul 06 22:25:35.720909 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.720824 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2sj22" event={"ID":"158c4ff9-36f2-454d-8082-94ab9b2e5bdb","Type":"ContainerStarted","Data":"ea444d0e798b2375c40150d34f7712deb99a2b2b0e9eb4b0610e15e93f70af23"} Jul 06 22:25:35.742647 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.741414 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2sj22" podStartSLOduration=66.521163129 podStartE2EDuration="1m7.741390149s" totalImagesPullingTime="1.22022702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:35.740093873 +0000 UTC m=+68.044262534" watchObservedRunningTime="2026-07-06 22:25:35.741390149 +0000 UTC m=+68.045558809" Jul 06 22:25:35.819604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:35.819507 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" Jul 06 22:25:36.456673 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.455813 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 22:25:36.462296 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.460659 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.464881 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.463872 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 06 22:25:36.464881 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.464207 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 06 22:25:36.464881 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.464449 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 06 22:25:36.464881 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.464671 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 06 22:25:36.465163 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.464900 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-a63gdjbf02l9v\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.467389 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.468723 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.468966 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.469200 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.469468 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.469709 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.469923 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-p8q8j\"" Jul 06 22:25:36.470103 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.470101 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 06 22:25:36.470585 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.470330 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 06 22:25:36.477422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.475660 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 06 22:25:36.477422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.475700 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 06 22:25:36.477422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.477389 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 22:25:36.507567 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507512 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507567 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507561 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-config\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507587 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507620 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507671 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507699 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507727 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507749 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.507812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507778 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507905 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507935 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f96da12e-6cde-49ae-b56d-1b01754d3289-config-out\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507963 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.507993 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508021 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-web-config\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508073 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zgh\" (UniqueName: \"kubernetes.io/projected/f96da12e-6cde-49ae-b56d-1b01754d3289-kube-api-access-s2zgh\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508099 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508147 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508133 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508180 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f96da12e-6cde-49ae-b56d-1b01754d3289-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508213 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.508521 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.508297 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.608858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.608826 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.608865 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.608893 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-config\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.608912 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609264 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609336 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609389 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609351 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609389 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609381 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609485 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609403 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609485 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609485 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609474 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609650 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609506 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f96da12e-6cde-49ae-b56d-1b01754d3289-config-out\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609650 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609650 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609554 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609650 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609576 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-web-config\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609650 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609598 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zgh\" (UniqueName: \"kubernetes.io/projected/f96da12e-6cde-49ae-b56d-1b01754d3289-kube-api-access-s2zgh\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609650 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609621 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609935 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609655 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609935 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609703 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f96da12e-6cde-49ae-b56d-1b01754d3289-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.609935 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.609736 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.610195 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.610177 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.610813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.610789 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.612266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.611923 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.612266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.612014 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.612690 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.612664 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.614277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.613407 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.618701 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.617666 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.618701 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.618657 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f96da12e-6cde-49ae-b56d-1b01754d3289-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.620665 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.619464 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f96da12e-6cde-49ae-b56d-1b01754d3289-config-out\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.620665 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.619781 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.620665 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.620106 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f96da12e-6cde-49ae-b56d-1b01754d3289-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.620665 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.620216 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.620665 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.620623 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.620969 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.620665 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.621486 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.621126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-config\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.621486 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.621436 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.621802 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.621751 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zgh\" (UniqueName: \"kubernetes.io/projected/f96da12e-6cde-49ae-b56d-1b01754d3289-kube-api-access-s2zgh\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.621879 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.621775 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.622338 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.622315 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-web-config\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.623557 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.623536 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f96da12e-6cde-49ae-b56d-1b01754d3289-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f96da12e-6cde-49ae-b56d-1b01754d3289\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:36.783068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:36.782701 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:39.608046 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:39.607727 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc"] Jul 06 22:25:39.876461 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:39.876383 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188689f4_7e55_4876_87bc_0bc264dc6205.slice/crio-36037419ef659c89c503a1471eadf79dc156cacae5d13369dd1faf13d6c1c0e4 WatchSource:0}: Error finding container 36037419ef659c89c503a1471eadf79dc156cacae5d13369dd1faf13d6c1c0e4: Status 404 returned error can't find the container with id 36037419ef659c89c503a1471eadf79dc156cacae5d13369dd1faf13d6c1c0e4 Jul 06 22:25:40.069263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.069156 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 06 22:25:40.086919 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:40.086884 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf96da12e_6cde_49ae_b56d_1b01754d3289.slice/crio-09e02740bbdf27254276548fdca78068b060f92e3b8db3c830b0446c2257dcad WatchSource:0}: Error finding container 09e02740bbdf27254276548fdca78068b060f92e3b8db3c830b0446c2257dcad: Status 404 returned error can't find the container with id 09e02740bbdf27254276548fdca78068b060f92e3b8db3c830b0446c2257dcad Jul 06 22:25:40.091206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.089145 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cc97cb67-qp87k"] Jul 06 22:25:40.096721 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:40.096475 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod704d460e_28a7_418f_88e8_582535c76815.slice/crio-5bc4acdd4aeca6475c82a6989870480c743e2d19b7eb53f9c2fb38e1dc4e3755 WatchSource:0}: Error finding container 5bc4acdd4aeca6475c82a6989870480c743e2d19b7eb53f9c2fb38e1dc4e3755: Status 404 returned error can't find the container with id 5bc4acdd4aeca6475c82a6989870480c743e2d19b7eb53f9c2fb38e1dc4e3755 Jul 06 22:25:40.744628 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.744588 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"62d964f837dda57bae111d7b80f42c8c03ed060218937ddcf87d0bdb495a4549"} Jul 06 22:25:40.745092 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.744700 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"9ea5e2e8cd22eb1cc76863e3130ce2fff5044489d079afc424df2f1db1a25f10"} Jul 06 22:25:40.745092 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.744744 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"7505c2b603a207d32fe16680cb1cb064fe1e5b0e282dc2277ac100865660d4da"} Jul 06 22:25:40.746176 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.746148 2572 generic.go:356] "Generic (PLEG): container finished" podID="f96da12e-6cde-49ae-b56d-1b01754d3289" containerID="2ede7e49c98649467c8629d1c54588353510cfcde094a2154d921ed4ef53a513" exitCode=0 Jul 06 22:25:40.746316 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.746255 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerDied","Data":"2ede7e49c98649467c8629d1c54588353510cfcde094a2154d921ed4ef53a513"} Jul 06 22:25:40.746430 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.746343 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"09e02740bbdf27254276548fdca78068b060f92e3b8db3c830b0446c2257dcad"} Jul 06 22:25:40.747712 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.747692 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-vlvg2" event={"ID":"8b4cf0a7-678c-4f21-8d45-a46b9e1dd522","Type":"ContainerStarted","Data":"3b9c64442673e8a9aa7e679147691fc25f7677cfd6000145dea60eb746f5af61"} Jul 06 22:25:40.749430 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.749409 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xrmgf" event={"ID":"361ac06c-1abe-49fd-bd1a-4f8083a9e962","Type":"ContainerStarted","Data":"4abfaed0332d12d9191eba9b88b55a636c13c6fa9eab643d9536522f4074d85e"} Jul 06 22:25:40.749654 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.749456 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:25:40.751059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.751039 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc97cb67-qp87k" event={"ID":"704d460e-28a7-418f-88e8-582535c76815","Type":"ContainerStarted","Data":"ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c"} Jul 06 22:25:40.751164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.751076 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc97cb67-qp87k" event={"ID":"704d460e-28a7-418f-88e8-582535c76815","Type":"ContainerStarted","Data":"5bc4acdd4aeca6475c82a6989870480c743e2d19b7eb53f9c2fb38e1dc4e3755"} Jul 06 22:25:40.754202 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.754185 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"94f26bbd93aa1fbfd97dffed949c0ed45c792547d2df53fcda2a77f169d615ef"} Jul 06 22:25:40.754315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.754257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"5771e0e56f3e9514b8487481f1df28a466d7aed945384518cf743b4d957568a2"} Jul 06 22:25:40.754315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.754287 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"3e3a02ad10546ba5d779a9abfda7175fe3fcfa6276e5ff3d0acaa407c9fe7401"} Jul 06 22:25:40.754397 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.754317 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"4b54068613393c9d74367f53b45dcce9dab3b1df8078e7ea896c44ef61b82112"} Jul 06 22:25:40.754397 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.754347 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"ca64df8ce8056b86d2a3e3b01a3a0b9176bf2e06959a67278fa4ebe0f81f704b"} Jul 06 22:25:40.755587 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.755569 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" event={"ID":"d3d957cc-6cf8-4f15-a836-1f8392b79312","Type":"ContainerStarted","Data":"22168a07f0b91cb69b466b33b5a23320e3d67f353b3456719f3e403752d6b281"} Jul 06 22:25:40.756675 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.756657 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" event={"ID":"188689f4-7e55-4876-87bc-0bc264dc6205","Type":"ContainerStarted","Data":"36037419ef659c89c503a1471eadf79dc156cacae5d13369dd1faf13d6c1c0e4"} Jul 06 22:25:40.790991 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.790926 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" podStartSLOduration=2.568440658 podStartE2EDuration="6.790909333s" totalImagesPullingTime="4.222468675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:40.790869622 +0000 UTC m=+73.095038292" watchObservedRunningTime="2026-07-06 22:25:40.790909333 +0000 UTC m=+73.095078003" Jul 06 22:25:40.809762 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.809687 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cc97cb67-qp87k" podStartSLOduration=6.809664963 podStartE2EDuration="6.809664963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:40.809128251 +0000 UTC m=+73.113296912" watchObservedRunningTime="2026-07-06 22:25:40.809664963 +0000 UTC m=+73.113833621" Jul 06 22:25:40.828507 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.828448 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xrmgf" podStartSLOduration=66.492199001 podStartE2EDuration="1m12.82842898s" totalImagesPullingTime="6.336229979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:40.827914184 +0000 UTC m=+73.132082834" watchObservedRunningTime="2026-07-06 22:25:40.82842898 +0000 UTC m=+73.132597638" Jul 06 22:25:40.849695 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:40.849635 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-vlvg2" podStartSLOduration=66.459586081 podStartE2EDuration="1m12.849612488s" totalImagesPullingTime="6.390026407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:40.848378065 +0000 UTC m=+73.152546726" watchObservedRunningTime="2026-07-06 22:25:40.849612488 +0000 UTC m=+73.153781147" Jul 06 22:25:41.768327 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.767992 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3b2d387e-c01a-4b0f-8eea-6425ed5e8d42","Type":"ContainerStarted","Data":"fb3ca3c8aeeeff37245c6410eb0d6f68c674f8adc697f3eb980971c667eddde2"} Jul 06 22:25:41.773378 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.772538 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"35993e13f0b4b98eee0b0bb39c7ebfee2d716cfb99a44367fc9f77d36b0a32dd"} Jul 06 22:25:41.773378 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.772610 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"5eddd168b296a4a9dd03c9a1131ed8fb74004ba6f99b75f83e451f2b128d25f7"} Jul 06 22:25:41.773378 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.772652 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" event={"ID":"d77792fd-44c9-4ebb-ab23-0e690f960164","Type":"ContainerStarted","Data":"6446da6697688c45817800fa38b3181d8ab31924226f6ab14b77685261018cb1"} Jul 06 22:25:41.773378 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.773304 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:41.797364 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.797310 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.180407898 podStartE2EDuration="10.797290595s" totalImagesPullingTime="7.593309625s" totalInitContainerRuntime="23.573072ms" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:41.794401276 +0000 UTC m=+74.098569937" watchObservedRunningTime="2026-07-06 22:25:41.797290595 +0000 UTC m=+74.101459258" Jul 06 22:25:41.817221 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:41.817172 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" podStartSLOduration=1.6162444150000002 podStartE2EDuration="8.81715125s" totalImagesPullingTime="7.200906835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:41.81545318 +0000 UTC m=+74.119621840" watchObservedRunningTime="2026-07-06 22:25:41.81715125 +0000 UTC m=+74.121319975" Jul 06 22:25:42.779368 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:42.779332 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" event={"ID":"188689f4-7e55-4876-87bc-0bc264dc6205","Type":"ContainerStarted","Data":"c1c9e1ffad9ea1d756ea8c4f29891fccecd0065453ed134f68197e874e521b38"} Jul 06 22:25:42.779797 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:42.779392 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" event={"ID":"188689f4-7e55-4876-87bc-0bc264dc6205","Type":"ContainerStarted","Data":"a658ba058e73ad0cc4c5471dd78449697ff054f9c670468072d07b5ecc9c4bfd"} Jul 06 22:25:42.779797 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:42.779412 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" event={"ID":"188689f4-7e55-4876-87bc-0bc264dc6205","Type":"ContainerStarted","Data":"8683479c2ca66cc6455041b071aa2c0607b8afa6e1d8a75c0456d9574102bf0c"} Jul 06 22:25:42.803718 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:42.803668 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6b6c44fc4d-fvlwc" podStartSLOduration=5.629313402 podStartE2EDuration="7.803653385s" totalImagesPullingTime="2.174339983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:42.801570451 +0000 UTC m=+75.105739109" watchObservedRunningTime="2026-07-06 22:25:42.803653385 +0000 UTC m=+75.107822056" Jul 06 22:25:43.522226 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.522188 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5665b66699-ph4z8"] Jul 06 22:25:43.563127 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.563093 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66df9cbc8d-bm9m7"] Jul 06 22:25:43.569746 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.569722 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.579862 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.579832 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66df9cbc8d-bm9m7"] Jul 06 22:25:43.681277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681224 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-service-ca\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.681277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681284 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-oauth-config\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.681512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681305 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-config\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.681512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681353 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdz47\" (UniqueName: \"kubernetes.io/projected/50701b5f-43e7-48ea-875d-1e3215d95c2d-kube-api-access-hdz47\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.681512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681403 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-oauth-serving-cert\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.681512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681471 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-trusted-ca-bundle\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.681664 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.681567 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-serving-cert\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782436 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-serving-cert\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782436 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782409 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-service-ca\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782436 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782427 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-oauth-config\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782987 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782451 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-config\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782987 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782475 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdz47\" (UniqueName: \"kubernetes.io/projected/50701b5f-43e7-48ea-875d-1e3215d95c2d-kube-api-access-hdz47\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782987 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782493 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-oauth-serving-cert\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.782987 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.782527 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-trusted-ca-bundle\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.783509 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.783486 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-trusted-ca-bundle\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.784035 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.784013 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-config\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.784853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.784834 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-oauth-serving-cert\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.785692 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.785661 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-service-ca\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.787157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.787131 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-oauth-config\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.787559 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.787538 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-serving-cert\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.793399 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.793376 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdz47\" (UniqueName: \"kubernetes.io/projected/50701b5f-43e7-48ea-875d-1e3215d95c2d-kube-api-access-hdz47\") pod \"console-66df9cbc8d-bm9m7\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:43.885167 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:43.885088 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:44.142347 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.142325 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66df9cbc8d-bm9m7"] Jul 06 22:25:44.144668 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:25:44.144643 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50701b5f_43e7_48ea_875d_1e3215d95c2d.slice/crio-86e5f8c32890852da06d6b8e94fbcccd2b85d566de9a2e25c22dedb5207fc2cb WatchSource:0}: Error finding container 86e5f8c32890852da06d6b8e94fbcccd2b85d566de9a2e25c22dedb5207fc2cb: Status 404 returned error can't find the container with id 86e5f8c32890852da06d6b8e94fbcccd2b85d566de9a2e25c22dedb5207fc2cb Jul 06 22:25:44.796351 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.796282 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66df9cbc8d-bm9m7" event={"ID":"50701b5f-43e7-48ea-875d-1e3215d95c2d","Type":"ContainerStarted","Data":"8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4"} Jul 06 22:25:44.796798 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.796361 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66df9cbc8d-bm9m7" event={"ID":"50701b5f-43e7-48ea-875d-1e3215d95c2d","Type":"ContainerStarted","Data":"86e5f8c32890852da06d6b8e94fbcccd2b85d566de9a2e25c22dedb5207fc2cb"} Jul 06 22:25:44.800250 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.800215 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"5ab0e646c42c583a83843876607fe39b2b03db660d017098e2ae4db47b748514"} Jul 06 22:25:44.800385 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.800313 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"b02a184d5033cdec3d6214e112076e477a4b298f5036de3d39211b0abcb30e05"} Jul 06 22:25:44.800385 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.800366 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"fcd43a3865653ccad6e6a1bdb8b8cbf3c4a66d76c9e9b37e0bd13bf8f99b51d4"} Jul 06 22:25:44.800489 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.800408 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"b217437850006dcf98c3fa9652c95b0d44b2050ecc5c57485a2c21ed82ca7d54"} Jul 06 22:25:44.800489 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.800448 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"1082d66e193ebc3dde92e752316e70f5e20f7df739ddce2c55f907ff26ae3a6e"} Jul 06 22:25:44.800489 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.800486 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f96da12e-6cde-49ae-b56d-1b01754d3289","Type":"ContainerStarted","Data":"7ae58a87c369383efcbf12cab9abc43588343e748e8058eb50e5f4b85c2caf47"} Jul 06 22:25:44.816064 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.816021 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66df9cbc8d-bm9m7" podStartSLOduration=1.816007121 podStartE2EDuration="1.816007121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:44.814823321 +0000 UTC m=+77.118991985" watchObservedRunningTime="2026-07-06 22:25:44.816007121 +0000 UTC m=+77.120175780" Jul 06 22:25:44.845642 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:44.845042 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=5.490759841 podStartE2EDuration="8.845021907s" totalImagesPullingTime="3.326744861s" totalInitContainerRuntime="27.517205ms" isStatefulPod=false podCreationTimestamp="2026-07-06 22:25:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:25:44.84404238 +0000 UTC m=+77.148211038" watchObservedRunningTime="2026-07-06 22:25:44.845021907 +0000 UTC m=+77.149190567" Jul 06 22:25:45.306838 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:45.306791 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:45.307044 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:45.306864 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:45.311613 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:45.311590 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:45.812813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:45.812784 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:25:46.784030 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:46.783994 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:25:48.795652 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:48.795539 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7c684f54f5-bssqq" Jul 06 22:25:53.885671 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:53.885632 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:53.886256 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:53.885938 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:53.890536 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:53.890521 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:54.847199 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:54.847172 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:25:54.895351 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:54.895321 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc97cb67-qp87k"] Jul 06 22:25:54.957907 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:54.957866 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:54.958096 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:54.957940 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:25:59.989753 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:25:59.989691 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-dc64cd665-vtzsp" podUID="c5134b33-7704-463c-8734-9c6021194d78" containerName="console" containerID="cri-o://7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec" gracePeriod=15 Jul 06 22:26:00.737984 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.737960 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:26:00.743989 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.743966 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-oauth-serving-cert\") pod \"c5134b33-7704-463c-8734-9c6021194d78\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " Jul 06 22:26:00.744093 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.744009 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-service-ca\") pod \"c5134b33-7704-463c-8734-9c6021194d78\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " Jul 06 22:26:00.744093 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.744056 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-console-config\") pod \"c5134b33-7704-463c-8734-9c6021194d78\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " Jul 06 22:26:00.744173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.744127 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-oauth-config\") pod \"c5134b33-7704-463c-8734-9c6021194d78\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " Jul 06 22:26:00.744472 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.744377 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c5134b33-7704-463c-8734-9c6021194d78" (UID: "c5134b33-7704-463c-8734-9c6021194d78"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:00.744472 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.744462 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-service-ca" (OuterVolumeSpecName: "service-ca") pod "c5134b33-7704-463c-8734-9c6021194d78" (UID: "c5134b33-7704-463c-8734-9c6021194d78"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:00.744582 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.744544 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-console-config" (OuterVolumeSpecName: "console-config") pod "c5134b33-7704-463c-8734-9c6021194d78" (UID: "c5134b33-7704-463c-8734-9c6021194d78"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:00.746420 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.746395 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c5134b33-7704-463c-8734-9c6021194d78" (UID: "c5134b33-7704-463c-8734-9c6021194d78"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:26:00.844845 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.844814 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d98rv\" (UniqueName: \"kubernetes.io/projected/c5134b33-7704-463c-8734-9c6021194d78-kube-api-access-d98rv\") pod \"c5134b33-7704-463c-8734-9c6021194d78\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " Jul 06 22:26:00.844845 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.844848 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-serving-cert\") pod \"c5134b33-7704-463c-8734-9c6021194d78\" (UID: \"c5134b33-7704-463c-8734-9c6021194d78\") " Jul 06 22:26:00.845084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.845006 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-oauth-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:00.845084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.845017 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:00.845084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.845028 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c5134b33-7704-463c-8734-9c6021194d78-console-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:00.845084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.845036 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-oauth-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:00.847021 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.846989 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5134b33-7704-463c-8734-9c6021194d78-kube-api-access-d98rv" (OuterVolumeSpecName: "kube-api-access-d98rv") pod "c5134b33-7704-463c-8734-9c6021194d78" (UID: "c5134b33-7704-463c-8734-9c6021194d78"). InnerVolumeSpecName "kube-api-access-d98rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:00.847133 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.847101 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c5134b33-7704-463c-8734-9c6021194d78" (UID: "c5134b33-7704-463c-8734-9c6021194d78"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:26:00.872058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.871993 2572 generic.go:356] "Generic (PLEG): container finished" podID="c5134b33-7704-463c-8734-9c6021194d78" containerID="7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec" exitCode=0 Jul 06 22:26:00.872058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.872037 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dc64cd665-vtzsp" Jul 06 22:26:00.872173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.872079 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc64cd665-vtzsp" event={"ID":"c5134b33-7704-463c-8734-9c6021194d78","Type":"ContainerDied","Data":"7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec"} Jul 06 22:26:00.872173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.872153 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dc64cd665-vtzsp" event={"ID":"c5134b33-7704-463c-8734-9c6021194d78","Type":"ContainerDied","Data":"dbcabfff54d4b62abcc051d7df3c4c4cbfadd85080e8735c1f8ffa0ed1ea0d63"} Jul 06 22:26:00.872280 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.872187 2572 scope.go:117] "RemoveContainer" containerID="7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec" Jul 06 22:26:00.881597 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.881581 2572 scope.go:117] "RemoveContainer" containerID="7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec" Jul 06 22:26:00.881851 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:26:00.881831 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec\": container with ID starting with 7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec not found: ID does not exist" containerID="7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec" Jul 06 22:26:00.881890 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.881860 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec"} err="failed to get container status \"7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec\": rpc error: code = NotFound desc = could not find container \"7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec\": container with ID starting with 7a1f46d74b8d4bcaee541deebf97115bb3f0f833bf032d389f3a197cedc896ec not found: ID does not exist" Jul 06 22:26:00.893431 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.893401 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-dc64cd665-vtzsp"] Jul 06 22:26:00.894787 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.894769 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-dc64cd665-vtzsp"] Jul 06 22:26:00.945977 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.945952 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d98rv\" (UniqueName: \"kubernetes.io/projected/c5134b33-7704-463c-8734-9c6021194d78-kube-api-access-d98rv\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:00.946068 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:00.945980 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c5134b33-7704-463c-8734-9c6021194d78-console-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:02.247713 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:02.247679 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5134b33-7704-463c-8734-9c6021194d78" path="/var/lib/kubelet/pods/c5134b33-7704-463c-8734-9c6021194d78/volumes" Jul 06 22:26:08.550827 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.550790 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5665b66699-ph4z8" podUID="f902de25-dcb2-4093-b95a-216256d78d98" containerName="console" containerID="cri-o://f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f" gracePeriod=15 Jul 06 22:26:08.847507 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.847477 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:26:08.908222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.908191 2572 generic.go:356] "Generic (PLEG): container finished" podID="f902de25-dcb2-4093-b95a-216256d78d98" containerID="f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f" exitCode=0 Jul 06 22:26:08.908398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.908260 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5665b66699-ph4z8" Jul 06 22:26:08.908398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.908277 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b66699-ph4z8" event={"ID":"f902de25-dcb2-4093-b95a-216256d78d98","Type":"ContainerDied","Data":"f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f"} Jul 06 22:26:08.908398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.908363 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5665b66699-ph4z8" event={"ID":"f902de25-dcb2-4093-b95a-216256d78d98","Type":"ContainerDied","Data":"35d8f0a184edca3fb2d4c76b600fb69b38dab9336f955b1915183ab74fc9bbbd"} Jul 06 22:26:08.908398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.908384 2572 scope.go:117] "RemoveContainer" containerID="f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f" Jul 06 22:26:08.909476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909324 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-trusted-ca-bundle\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909378 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-oauth-serving-cert\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909401 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-console-config\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909425 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gztqj\" (UniqueName: \"kubernetes.io/projected/f902de25-dcb2-4093-b95a-216256d78d98-kube-api-access-gztqj\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909457 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-serving-cert\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909803 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909493 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-oauth-config\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909803 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909532 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-service-ca\") pod \"f902de25-dcb2-4093-b95a-216256d78d98\" (UID: \"f902de25-dcb2-4093-b95a-216256d78d98\") " Jul 06 22:26:08.909803 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909709 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:08.909803 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909768 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:08.909803 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.909774 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-console-config" (OuterVolumeSpecName: "console-config") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:08.910121 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.910029 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-service-ca" (OuterVolumeSpecName: "service-ca") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:08.912149 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.912120 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f902de25-dcb2-4093-b95a-216256d78d98-kube-api-access-gztqj" (OuterVolumeSpecName: "kube-api-access-gztqj") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "kube-api-access-gztqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:08.912149 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.912130 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:26:08.912344 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.912204 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f902de25-dcb2-4093-b95a-216256d78d98" (UID: "f902de25-dcb2-4093-b95a-216256d78d98"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:26:08.927921 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.927900 2572 scope.go:117] "RemoveContainer" containerID="f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f" Jul 06 22:26:08.928278 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:26:08.928255 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f\": container with ID starting with f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f not found: ID does not exist" containerID="f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f" Jul 06 22:26:08.928348 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:08.928286 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f"} err="failed to get container status \"f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f\": rpc error: code = NotFound desc = could not find container \"f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f\": container with ID starting with f395e0df90ca36c2fabe99212351f61eb68e78e388e9c06d95b425477340c17f not found: ID does not exist" Jul 06 22:26:09.011041 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011004 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-trusted-ca-bundle\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.011041 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011035 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-oauth-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.011041 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011047 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-console-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.011313 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011055 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gztqj\" (UniqueName: \"kubernetes.io/projected/f902de25-dcb2-4093-b95a-216256d78d98-kube-api-access-gztqj\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.011313 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011065 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.011313 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011074 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f902de25-dcb2-4093-b95a-216256d78d98-console-oauth-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.011313 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.011083 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f902de25-dcb2-4093-b95a-216256d78d98-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:09.230059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.230015 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5665b66699-ph4z8"] Jul 06 22:26:09.231682 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:09.231658 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5665b66699-ph4z8"] Jul 06 22:26:10.246981 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:10.246952 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f902de25-dcb2-4093-b95a-216256d78d98" path="/var/lib/kubelet/pods/f902de25-dcb2-4093-b95a-216256d78d98/volumes" Jul 06 22:26:11.775439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:11.775410 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xrmgf" Jul 06 22:26:14.970903 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:14.970865 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:26:14.976573 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:14.976545 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c86cd7fb9-l6rpf" Jul 06 22:26:19.916834 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:19.916790 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7cc97cb67-qp87k" podUID="704d460e-28a7-418f-88e8-582535c76815" containerName="console" containerID="cri-o://ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c" gracePeriod=15 Jul 06 22:26:20.161821 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.161798 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:26:20.218020 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.217990 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-console-config\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218165 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218032 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-oauth-serving-cert\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218165 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218052 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-service-ca\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218165 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218069 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-serving-cert\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218165 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218132 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-oauth-config\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218164 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgcdf\" (UniqueName: \"kubernetes.io/projected/704d460e-28a7-418f-88e8-582535c76815-kube-api-access-dgcdf\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218200 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-trusted-ca-bundle\") pod \"704d460e-28a7-418f-88e8-582535c76815\" (UID: \"704d460e-28a7-418f-88e8-582535c76815\") " Jul 06 22:26:20.218472 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218442 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:20.218527 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218502 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-service-ca" (OuterVolumeSpecName: "service-ca") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:20.218582 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218564 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-oauth-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.218746 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218719 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-console-config" (OuterVolumeSpecName: "console-config") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:20.218853 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.218726 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:26:20.220504 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.220468 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:26:20.220616 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.220565 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:26:20.220616 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.220584 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/704d460e-28a7-418f-88e8-582535c76815-kube-api-access-dgcdf" (OuterVolumeSpecName: "kube-api-access-dgcdf") pod "704d460e-28a7-418f-88e8-582535c76815" (UID: "704d460e-28a7-418f-88e8-582535c76815"). InnerVolumeSpecName "kube-api-access-dgcdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:26:20.319277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.319249 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-oauth-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.319277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.319272 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgcdf\" (UniqueName: \"kubernetes.io/projected/704d460e-28a7-418f-88e8-582535c76815-kube-api-access-dgcdf\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.319439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.319282 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-trusted-ca-bundle\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.319439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.319291 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-console-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.319439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.319300 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/704d460e-28a7-418f-88e8-582535c76815-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.319439 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.319308 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/704d460e-28a7-418f-88e8-582535c76815-console-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:26:20.970856 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.970818 2572 generic.go:356] "Generic (PLEG): container finished" podID="704d460e-28a7-418f-88e8-582535c76815" containerID="ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c" exitCode=0 Jul 06 22:26:20.971301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.970867 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cc97cb67-qp87k" Jul 06 22:26:20.971301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.970864 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc97cb67-qp87k" event={"ID":"704d460e-28a7-418f-88e8-582535c76815","Type":"ContainerDied","Data":"ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c"} Jul 06 22:26:20.971301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.970999 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cc97cb67-qp87k" event={"ID":"704d460e-28a7-418f-88e8-582535c76815","Type":"ContainerDied","Data":"5bc4acdd4aeca6475c82a6989870480c743e2d19b7eb53f9c2fb38e1dc4e3755"} Jul 06 22:26:20.971301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.971030 2572 scope.go:117] "RemoveContainer" containerID="ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c" Jul 06 22:26:20.979705 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.979688 2572 scope.go:117] "RemoveContainer" containerID="ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c" Jul 06 22:26:20.979966 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:26:20.979946 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c\": container with ID starting with ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c not found: ID does not exist" containerID="ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c" Jul 06 22:26:20.980026 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.979972 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c"} err="failed to get container status \"ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c\": rpc error: code = NotFound desc = could not find container \"ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c\": container with ID starting with ab95d5032f1276adc2c4cf210dc00ff2e10654dc029d271c36c0cc6b1c67817c not found: ID does not exist" Jul 06 22:26:20.988620 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.988598 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cc97cb67-qp87k"] Jul 06 22:26:20.991385 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:20.991363 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cc97cb67-qp87k"] Jul 06 22:26:22.247029 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:22.246992 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="704d460e-28a7-418f-88e8-582535c76815" path="/var/lib/kubelet/pods/704d460e-28a7-418f-88e8-582535c76815/volumes" Jul 06 22:26:36.783931 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:36.783889 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:26:36.803953 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:36.803926 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:26:37.055664 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:37.055587 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 06 22:26:51.168878 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.168842 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-588644757d-5ztqh"] Jul 06 22:26:51.169259 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169155 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="704d460e-28a7-418f-88e8-582535c76815" containerName="console" Jul 06 22:26:51.169259 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169167 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="704d460e-28a7-418f-88e8-582535c76815" containerName="console" Jul 06 22:26:51.169259 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169184 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f902de25-dcb2-4093-b95a-216256d78d98" containerName="console" Jul 06 22:26:51.169259 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169189 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f902de25-dcb2-4093-b95a-216256d78d98" containerName="console" Jul 06 22:26:51.169259 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169197 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5134b33-7704-463c-8734-9c6021194d78" containerName="console" Jul 06 22:26:51.169259 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169203 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5134b33-7704-463c-8734-9c6021194d78" containerName="console" Jul 06 22:26:51.169453 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169280 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="704d460e-28a7-418f-88e8-582535c76815" containerName="console" Jul 06 22:26:51.169453 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169291 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5134b33-7704-463c-8734-9c6021194d78" containerName="console" Jul 06 22:26:51.169453 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.169298 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f902de25-dcb2-4093-b95a-216256d78d98" containerName="console" Jul 06 22:26:51.171078 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.171061 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.185135 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.185113 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588644757d-5ztqh"] Jul 06 22:26:51.203710 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203684 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-service-ca\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.203808 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203728 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-trusted-ca-bundle\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.203808 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203750 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-config\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.203881 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203826 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-serving-cert\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.203920 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203883 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-oauth-config\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.203954 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203918 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lslk6\" (UniqueName: \"kubernetes.io/projected/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-kube-api-access-lslk6\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.203988 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.203956 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-oauth-serving-cert\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.305333 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.305288 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-oauth-serving-cert\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.305512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.305358 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-service-ca\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.305512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.305410 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-trusted-ca-bundle\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.305512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.305436 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-config\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.305512 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.305494 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-serving-cert\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.306285 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-oauth-config\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.306340 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lslk6\" (UniqueName: \"kubernetes.io/projected/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-kube-api-access-lslk6\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.306416 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-oauth-serving-cert\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.306715 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-config\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.306715 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-service-ca\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.308412 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-trusted-ca-bundle\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.309489 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-oauth-config\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.311854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.310821 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-serving-cert\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.315178 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.315155 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lslk6\" (UniqueName: \"kubernetes.io/projected/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-kube-api-access-lslk6\") pod \"console-588644757d-5ztqh\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.480455 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.480416 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:26:51.607222 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:51.607196 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-588644757d-5ztqh"] Jul 06 22:26:51.609549 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:26:51.609521 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf36eba92_e5f8_486d_ae4b_6d7d7f288da4.slice/crio-dfa8c2e41f7970a7bdbefcaf7c16512d457305d0f277a85cfd411030b87b5118 WatchSource:0}: Error finding container dfa8c2e41f7970a7bdbefcaf7c16512d457305d0f277a85cfd411030b87b5118: Status 404 returned error can't find the container with id dfa8c2e41f7970a7bdbefcaf7c16512d457305d0f277a85cfd411030b87b5118 Jul 06 22:26:52.109125 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:52.109096 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588644757d-5ztqh" event={"ID":"f36eba92-e5f8-486d-ae4b-6d7d7f288da4","Type":"ContainerStarted","Data":"08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00"} Jul 06 22:26:52.109325 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:52.109147 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588644757d-5ztqh" event={"ID":"f36eba92-e5f8-486d-ae4b-6d7d7f288da4","Type":"ContainerStarted","Data":"dfa8c2e41f7970a7bdbefcaf7c16512d457305d0f277a85cfd411030b87b5118"} Jul 06 22:26:52.125476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:26:52.125427 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-588644757d-5ztqh" podStartSLOduration=1.125408229 podStartE2EDuration="1.125408229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:26:52.124850219 +0000 UTC m=+144.429018878" watchObservedRunningTime="2026-07-06 22:26:52.125408229 +0000 UTC m=+144.429576889" Jul 06 22:27:01.481460 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:01.481431 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:27:01.481924 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:01.481504 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:27:01.486190 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:01.486164 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:27:02.153452 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:02.153423 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:27:02.224981 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:02.224950 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66df9cbc8d-bm9m7"] Jul 06 22:27:27.253075 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.251647 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-66df9cbc8d-bm9m7" podUID="50701b5f-43e7-48ea-875d-1e3215d95c2d" containerName="console" containerID="cri-o://8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4" gracePeriod=15 Jul 06 22:27:27.497273 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.497251 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:27:27.532450 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532385 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdz47\" (UniqueName: \"kubernetes.io/projected/50701b5f-43e7-48ea-875d-1e3215d95c2d-kube-api-access-hdz47\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532573 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532518 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-oauth-serving-cert\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532632 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532574 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-service-ca\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532632 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532598 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-trusted-ca-bundle\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532729 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532630 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-serving-cert\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532729 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532676 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-oauth-config\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532822 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532738 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-config\") pod \"50701b5f-43e7-48ea-875d-1e3215d95c2d\" (UID: \"50701b5f-43e7-48ea-875d-1e3215d95c2d\") " Jul 06 22:27:27.532934 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.532906 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-service-ca" (OuterVolumeSpecName: "service-ca") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:27:27.533015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.533003 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:27.533073 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.533014 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:27:27.533274 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.533220 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-config" (OuterVolumeSpecName: "console-config") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:27:27.533386 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.533280 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:27:27.534868 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.534837 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:27:27.535072 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.535051 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50701b5f-43e7-48ea-875d-1e3215d95c2d-kube-api-access-hdz47" (OuterVolumeSpecName: "kube-api-access-hdz47") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "kube-api-access-hdz47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:27:27.535145 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.535128 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "50701b5f-43e7-48ea-875d-1e3215d95c2d" (UID: "50701b5f-43e7-48ea-875d-1e3215d95c2d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:27:27.633422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.633378 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:27.633422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.633417 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdz47\" (UniqueName: \"kubernetes.io/projected/50701b5f-43e7-48ea-875d-1e3215d95c2d-kube-api-access-hdz47\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:27.633422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.633428 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-oauth-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:27.633422 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.633437 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/50701b5f-43e7-48ea-875d-1e3215d95c2d-trusted-ca-bundle\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:27.633652 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.633445 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:27.633652 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:27.633453 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/50701b5f-43e7-48ea-875d-1e3215d95c2d-console-oauth-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:27:28.258065 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.258036 2572 generic.go:356] "Generic (PLEG): container finished" podID="50701b5f-43e7-48ea-875d-1e3215d95c2d" containerID="8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4" exitCode=0 Jul 06 22:27:28.258476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.258081 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66df9cbc8d-bm9m7" Jul 06 22:27:28.258476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.258089 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66df9cbc8d-bm9m7" event={"ID":"50701b5f-43e7-48ea-875d-1e3215d95c2d","Type":"ContainerDied","Data":"8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4"} Jul 06 22:27:28.258476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.258124 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66df9cbc8d-bm9m7" event={"ID":"50701b5f-43e7-48ea-875d-1e3215d95c2d","Type":"ContainerDied","Data":"86e5f8c32890852da06d6b8e94fbcccd2b85d566de9a2e25c22dedb5207fc2cb"} Jul 06 22:27:28.258476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.258142 2572 scope.go:117] "RemoveContainer" containerID="8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4" Jul 06 22:27:28.266174 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.266157 2572 scope.go:117] "RemoveContainer" containerID="8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4" Jul 06 22:27:28.266461 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:27:28.266443 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4\": container with ID starting with 8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4 not found: ID does not exist" containerID="8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4" Jul 06 22:27:28.266538 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.266470 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4"} err="failed to get container status \"8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4\": rpc error: code = NotFound desc = could not find container \"8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4\": container with ID starting with 8f341cebeaefb2797d79e09be71b5169e570ddead181446a0c5a5ae53d1e6fa4 not found: ID does not exist" Jul 06 22:27:28.274816 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.274792 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66df9cbc8d-bm9m7"] Jul 06 22:27:28.278031 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:28.278006 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66df9cbc8d-bm9m7"] Jul 06 22:27:30.247365 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:27:30.247335 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50701b5f-43e7-48ea-875d-1e3215d95c2d" path="/var/lib/kubelet/pods/50701b5f-43e7-48ea-875d-1e3215d95c2d/volumes" Jul 06 22:28:29.982318 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.982287 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w"] Jul 06 22:28:29.982714 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.982633 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50701b5f-43e7-48ea-875d-1e3215d95c2d" containerName="console" Jul 06 22:28:29.982714 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.982643 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="50701b5f-43e7-48ea-875d-1e3215d95c2d" containerName="console" Jul 06 22:28:29.982714 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.982705 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="50701b5f-43e7-48ea-875d-1e3215d95c2d" containerName="console" Jul 06 22:28:29.984907 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.984890 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:29.987747 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.987721 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:28:29.989189 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.989172 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 22:28:29.989277 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.989204 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cvb52\"" Jul 06 22:28:29.993149 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:29.993131 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w"] Jul 06 22:28:30.082550 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.082514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca086f1b-7b41-45dc-b153-1c61465c02b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nhs2w\" (UID: \"ca086f1b-7b41-45dc-b153-1c61465c02b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.082714 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.082569 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhqjj\" (UniqueName: \"kubernetes.io/projected/ca086f1b-7b41-45dc-b153-1c61465c02b3-kube-api-access-qhqjj\") pod \"openshift-lws-operator-fd8ccff4c-nhs2w\" (UID: \"ca086f1b-7b41-45dc-b153-1c61465c02b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.183258 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.183206 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhqjj\" (UniqueName: \"kubernetes.io/projected/ca086f1b-7b41-45dc-b153-1c61465c02b3-kube-api-access-qhqjj\") pod \"openshift-lws-operator-fd8ccff4c-nhs2w\" (UID: \"ca086f1b-7b41-45dc-b153-1c61465c02b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.183414 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.183309 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca086f1b-7b41-45dc-b153-1c61465c02b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nhs2w\" (UID: \"ca086f1b-7b41-45dc-b153-1c61465c02b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.183610 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.183594 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca086f1b-7b41-45dc-b153-1c61465c02b3-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nhs2w\" (UID: \"ca086f1b-7b41-45dc-b153-1c61465c02b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.191628 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.191608 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhqjj\" (UniqueName: \"kubernetes.io/projected/ca086f1b-7b41-45dc-b153-1c61465c02b3-kube-api-access-qhqjj\") pod \"openshift-lws-operator-fd8ccff4c-nhs2w\" (UID: \"ca086f1b-7b41-45dc-b153-1c61465c02b3\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.304959 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.304883 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" Jul 06 22:28:30.431334 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.431304 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w"] Jul 06 22:28:30.434455 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:28:30.434434 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca086f1b_7b41_45dc_b153_1c61465c02b3.slice/crio-c29c57cd7f73fe1549a49bf96370ad0cd71cb83ec800adea381900158bf595a8 WatchSource:0}: Error finding container c29c57cd7f73fe1549a49bf96370ad0cd71cb83ec800adea381900158bf595a8: Status 404 returned error can't find the container with id c29c57cd7f73fe1549a49bf96370ad0cd71cb83ec800adea381900158bf595a8 Jul 06 22:28:30.506333 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:30.506300 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" event={"ID":"ca086f1b-7b41-45dc-b153-1c61465c02b3","Type":"ContainerStarted","Data":"c29c57cd7f73fe1549a49bf96370ad0cd71cb83ec800adea381900158bf595a8"} Jul 06 22:28:33.523622 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:33.523584 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" event={"ID":"ca086f1b-7b41-45dc-b153-1c61465c02b3","Type":"ContainerStarted","Data":"d2abfbe80505d404a5894806a698190eef26e0e014573246dd48d957a898e805"} Jul 06 22:28:33.540600 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:33.540547 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nhs2w" podStartSLOduration=2.122793468 podStartE2EDuration="4.540532542s" totalImagesPullingTime="2.417739074s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:28:33.539073828 +0000 UTC m=+245.843242488" watchObservedRunningTime="2026-07-06 22:28:33.540532542 +0000 UTC m=+245.844701203" Jul 06 22:28:50.045978 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.045898 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52"] Jul 06 22:28:50.049207 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.049184 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.052588 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.052553 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n44fs\"" Jul 06 22:28:50.052710 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.052676 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 22:28:50.052982 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.052961 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 22:28:50.053085 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.053064 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 22:28:50.053149 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.053137 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 22:28:50.061728 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.061709 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghrkk\" (UniqueName: \"kubernetes.io/projected/e63e4f76-619a-4a3c-9292-e8e02ec385fa-kube-api-access-ghrkk\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.061830 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.061743 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e63e4f76-619a-4a3c-9292-e8e02ec385fa-manifests\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.061830 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.061762 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e63e4f76-619a-4a3c-9292-e8e02ec385fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.061830 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.061796 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63e4f76-619a-4a3c-9292-e8e02ec385fa-tmp\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.061929 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.061847 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e63e4f76-619a-4a3c-9292-e8e02ec385fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.069813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.069792 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52"] Jul 06 22:28:50.162694 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.162668 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghrkk\" (UniqueName: \"kubernetes.io/projected/e63e4f76-619a-4a3c-9292-e8e02ec385fa-kube-api-access-ghrkk\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.162871 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.162710 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e63e4f76-619a-4a3c-9292-e8e02ec385fa-manifests\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.162871 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.162728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e63e4f76-619a-4a3c-9292-e8e02ec385fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.162871 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.162747 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63e4f76-619a-4a3c-9292-e8e02ec385fa-tmp\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.162871 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.162806 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e63e4f76-619a-4a3c-9292-e8e02ec385fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.163139 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.163118 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e63e4f76-619a-4a3c-9292-e8e02ec385fa-manifests\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.165022 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.165001 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e63e4f76-619a-4a3c-9292-e8e02ec385fa-tmp\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.165347 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.165331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e63e4f76-619a-4a3c-9292-e8e02ec385fa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.165401 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.165355 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e63e4f76-619a-4a3c-9292-e8e02ec385fa-webhook-cert\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.176456 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.176435 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghrkk\" (UniqueName: \"kubernetes.io/projected/e63e4f76-619a-4a3c-9292-e8e02ec385fa-kube-api-access-ghrkk\") pod \"opendatahub-operator-controller-manager-84b8787b6c-gjb52\" (UID: \"e63e4f76-619a-4a3c-9292-e8e02ec385fa\") " pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.359442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.359362 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:50.525015 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.524991 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52"] Jul 06 22:28:50.528206 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:28:50.528179 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode63e4f76_619a_4a3c_9292_e8e02ec385fa.slice/crio-f31febedbc42d95b610cb6db6d6787fefb40e0fac5da756b5e5d1681011a0cbd WatchSource:0}: Error finding container f31febedbc42d95b610cb6db6d6787fefb40e0fac5da756b5e5d1681011a0cbd: Status 404 returned error can't find the container with id f31febedbc42d95b610cb6db6d6787fefb40e0fac5da756b5e5d1681011a0cbd Jul 06 22:28:50.597784 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:50.597738 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" event={"ID":"e63e4f76-619a-4a3c-9292-e8e02ec385fa","Type":"ContainerStarted","Data":"f31febedbc42d95b610cb6db6d6787fefb40e0fac5da756b5e5d1681011a0cbd"} Jul 06 22:28:54.619169 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:54.619135 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" event={"ID":"e63e4f76-619a-4a3c-9292-e8e02ec385fa","Type":"ContainerStarted","Data":"6dea5e53e6307ebfe7300b77cc3418ee1d80e1d2e60b7ba002a74d0321cfba0a"} Jul 06 22:28:55.624916 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:55.624879 2572 generic.go:356] "Generic (PLEG): container finished" podID="e63e4f76-619a-4a3c-9292-e8e02ec385fa" containerID="6dea5e53e6307ebfe7300b77cc3418ee1d80e1d2e60b7ba002a74d0321cfba0a" exitCode=0 Jul 06 22:28:55.625376 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:55.624971 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" event={"ID":"e63e4f76-619a-4a3c-9292-e8e02ec385fa","Type":"ContainerDied","Data":"6dea5e53e6307ebfe7300b77cc3418ee1d80e1d2e60b7ba002a74d0321cfba0a"} Jul 06 22:28:56.630998 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:56.630964 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" event={"ID":"e63e4f76-619a-4a3c-9292-e8e02ec385fa","Type":"ContainerStarted","Data":"b632ff70e49946c7f79460dcdf153b50b1857c4ca0de67c4efd53146e8cec6a2"} Jul 06 22:28:56.631437 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:56.631061 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:28:56.655354 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:28:56.655309 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" podStartSLOduration=1.932560899 podStartE2EDuration="6.655294366s" totalImagesPullingTime="3.62533775s" totalInitContainerRuntime="1.097395717s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:28:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:28:56.654790522 +0000 UTC m=+268.958959184" watchObservedRunningTime="2026-07-06 22:28:56.655294366 +0000 UTC m=+268.959463025" Jul 06 22:29:07.638506 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:07.638470 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84b8787b6c-gjb52" Jul 06 22:29:08.996623 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:08.996587 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q"] Jul 06 22:29:09.004805 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.004779 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.009991 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.009959 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mslft\"" Jul 06 22:29:09.010130 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.010033 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 06 22:29:09.010130 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.010069 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 06 22:29:09.010130 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.010121 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 06 22:29:09.021918 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.021887 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q"] Jul 06 22:29:09.138553 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.138514 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27a05754-4a61-4622-889d-6d2f68449e84-manager-config\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.138553 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.138552 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a05754-4a61-4622-889d-6d2f68449e84-cert\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.138769 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.138592 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkwzk\" (UniqueName: \"kubernetes.io/projected/27a05754-4a61-4622-889d-6d2f68449e84-kube-api-access-dkwzk\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.138769 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.138642 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27a05754-4a61-4622-889d-6d2f68449e84-metrics-cert\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.241529 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.241492 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkwzk\" (UniqueName: \"kubernetes.io/projected/27a05754-4a61-4622-889d-6d2f68449e84-kube-api-access-dkwzk\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.241707 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.241547 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27a05754-4a61-4622-889d-6d2f68449e84-metrics-cert\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.241707 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.241638 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27a05754-4a61-4622-889d-6d2f68449e84-manager-config\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.241707 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.241662 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a05754-4a61-4622-889d-6d2f68449e84-cert\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.242361 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.242336 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/27a05754-4a61-4622-889d-6d2f68449e84-manager-config\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.244174 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.244155 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/27a05754-4a61-4622-889d-6d2f68449e84-metrics-cert\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.244304 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.244287 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/27a05754-4a61-4622-889d-6d2f68449e84-cert\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.272018 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.271952 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkwzk\" (UniqueName: \"kubernetes.io/projected/27a05754-4a61-4622-889d-6d2f68449e84-kube-api-access-dkwzk\") pod \"lws-controller-manager-69b75b5694-shk4q\" (UID: \"27a05754-4a61-4622-889d-6d2f68449e84\") " pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.314565 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.314537 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:09.488377 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.488343 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q"] Jul 06 22:29:09.492100 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:29:09.492076 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a05754_4a61_4622_889d_6d2f68449e84.slice/crio-8a10936418e20f77c521d8a28d5331a6c9cf7236686326bb85f5c2be27a59a57 WatchSource:0}: Error finding container 8a10936418e20f77c521d8a28d5331a6c9cf7236686326bb85f5c2be27a59a57: Status 404 returned error can't find the container with id 8a10936418e20f77c521d8a28d5331a6c9cf7236686326bb85f5c2be27a59a57 Jul 06 22:29:09.688295 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:09.688261 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" event={"ID":"27a05754-4a61-4622-889d-6d2f68449e84","Type":"ContainerStarted","Data":"8a10936418e20f77c521d8a28d5331a6c9cf7236686326bb85f5c2be27a59a57"} Jul 06 22:29:10.153448 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.153368 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl"] Jul 06 22:29:10.157141 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.157119 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.161600 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.161575 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 22:29:10.161740 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.161579 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 22:29:10.161740 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.161626 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 22:29:10.162850 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.162832 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-tg4wb\"" Jul 06 22:29:10.162965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.162833 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 22:29:10.172332 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.172308 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl"] Jul 06 22:29:10.251315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.251275 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a00628a2-128d-4499-b556-f8ca7f43572b-tmp\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.251618 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.251323 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a00628a2-128d-4499-b556-f8ca7f43572b-tls-certs\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.251618 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.251364 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhfd\" (UniqueName: \"kubernetes.io/projected/a00628a2-128d-4499-b556-f8ca7f43572b-kube-api-access-5qhfd\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.352864 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.352822 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a00628a2-128d-4499-b556-f8ca7f43572b-tmp\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.352864 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.352868 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a00628a2-128d-4499-b556-f8ca7f43572b-tls-certs\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.353106 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.352907 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhfd\" (UniqueName: \"kubernetes.io/projected/a00628a2-128d-4499-b556-f8ca7f43572b-kube-api-access-5qhfd\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.355669 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.355639 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a00628a2-128d-4499-b556-f8ca7f43572b-tmp\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.356139 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.356116 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a00628a2-128d-4499-b556-f8ca7f43572b-tls-certs\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.377034 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.376999 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhfd\" (UniqueName: \"kubernetes.io/projected/a00628a2-128d-4499-b556-f8ca7f43572b-kube-api-access-5qhfd\") pod \"kube-auth-proxy-7c6b7b4fbb-m8lxl\" (UID: \"a00628a2-128d-4499-b556-f8ca7f43572b\") " pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.471037 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.471004 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" Jul 06 22:29:10.637370 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:10.637341 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl"] Jul 06 22:29:10.904210 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:29:10.904133 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00628a2_128d_4499_b556_f8ca7f43572b.slice/crio-c5964aa23daa7ba0c0ee380df6a5f1db2352e470f03c15098448504b62802024 WatchSource:0}: Error finding container c5964aa23daa7ba0c0ee380df6a5f1db2352e470f03c15098448504b62802024: Status 404 returned error can't find the container with id c5964aa23daa7ba0c0ee380df6a5f1db2352e470f03c15098448504b62802024 Jul 06 22:29:11.701212 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:11.701169 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" event={"ID":"a00628a2-128d-4499-b556-f8ca7f43572b","Type":"ContainerStarted","Data":"c5964aa23daa7ba0c0ee380df6a5f1db2352e470f03c15098448504b62802024"} Jul 06 22:29:11.703121 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:11.703066 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" event={"ID":"27a05754-4a61-4622-889d-6d2f68449e84","Type":"ContainerStarted","Data":"dfe08ab75c86aa4ea8bdcc90229e49482c5f495bac2a7e93c3a48c9ac2cb8af1"} Jul 06 22:29:11.703289 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:11.703212 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:11.724356 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:11.724311 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" podStartSLOduration=2.256971852 podStartE2EDuration="3.724296345s" totalImagesPullingTime="1.467324493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:11.72302932 +0000 UTC m=+284.027197980" watchObservedRunningTime="2026-07-06 22:29:11.724296345 +0000 UTC m=+284.028465004" Jul 06 22:29:13.715410 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:13.715371 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" event={"ID":"a00628a2-128d-4499-b556-f8ca7f43572b","Type":"ContainerStarted","Data":"0493c6bd79cffb2178f23d041ce4e792fea36f61bed9edcf5861c7ecb8ac93a2"} Jul 06 22:29:13.733798 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:13.733749 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c6b7b4fbb-m8lxl" podStartSLOduration=1.339010689 podStartE2EDuration="3.733734653s" totalImagesPullingTime="2.394723964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:13.732224478 +0000 UTC m=+286.036393140" watchObservedRunningTime="2026-07-06 22:29:13.733734653 +0000 UTC m=+286.037903311" Jul 06 22:29:21.662338 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.662305 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m"] Jul 06 22:29:21.666017 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.665992 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:21.671251 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.671218 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 22:29:21.671800 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.671780 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-pcxfg\"" Jul 06 22:29:21.686912 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.686882 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m"] Jul 06 22:29:21.756720 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.756689 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8gz7\" (UniqueName: \"kubernetes.io/projected/c9548ebb-ca06-4375-be3e-bb36a9add5de-kube-api-access-n8gz7\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:21.756895 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.756734 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9548ebb-ca06-4375-be3e-bb36a9add5de-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:21.857806 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.857766 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8gz7\" (UniqueName: \"kubernetes.io/projected/c9548ebb-ca06-4375-be3e-bb36a9add5de-kube-api-access-n8gz7\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:21.857949 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.857817 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9548ebb-ca06-4375-be3e-bb36a9add5de-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:21.858007 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:29:21.857990 2572 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 06 22:29:21.858064 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:29:21.858054 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9548ebb-ca06-4375-be3e-bb36a9add5de-cert podName:c9548ebb-ca06-4375-be3e-bb36a9add5de nodeName:}" failed. No retries permitted until 2026-07-06 22:29:22.358039576 +0000 UTC m=+294.662208217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c9548ebb-ca06-4375-be3e-bb36a9add5de-cert") pod "llmisvc-controller-manager-777c6bd7d5-h6l2m" (UID: "c9548ebb-ca06-4375-be3e-bb36a9add5de") : secret "llmisvc-webhook-server-cert" not found Jul 06 22:29:21.872496 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:21.872461 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8gz7\" (UniqueName: \"kubernetes.io/projected/c9548ebb-ca06-4375-be3e-bb36a9add5de-kube-api-access-n8gz7\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:22.363063 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:22.363023 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9548ebb-ca06-4375-be3e-bb36a9add5de-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:22.365750 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:22.365720 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9548ebb-ca06-4375-be3e-bb36a9add5de-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-h6l2m\" (UID: \"c9548ebb-ca06-4375-be3e-bb36a9add5de\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:22.576448 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:22.576411 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:22.716702 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:22.716671 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69b75b5694-shk4q" Jul 06 22:29:22.780369 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:22.780346 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m"] Jul 06 22:29:22.782767 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:29:22.782742 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc9548ebb_ca06_4375_be3e_bb36a9add5de.slice/crio-ace6a1b6187b8f283a091a7a31d4d5212ec28cd17eb05f49dafbf8ac12270577 WatchSource:0}: Error finding container ace6a1b6187b8f283a091a7a31d4d5212ec28cd17eb05f49dafbf8ac12270577: Status 404 returned error can't find the container with id ace6a1b6187b8f283a091a7a31d4d5212ec28cd17eb05f49dafbf8ac12270577 Jul 06 22:29:23.762593 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:23.762554 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" event={"ID":"c9548ebb-ca06-4375-be3e-bb36a9add5de","Type":"ContainerStarted","Data":"ace6a1b6187b8f283a091a7a31d4d5212ec28cd17eb05f49dafbf8ac12270577"} Jul 06 22:29:25.773406 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:25.773331 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" event={"ID":"c9548ebb-ca06-4375-be3e-bb36a9add5de","Type":"ContainerStarted","Data":"41df1caef9283b09bc6d8141cb2258fb9ecef8e8d948bfe247bf444ca8fddc9a"} Jul 06 22:29:25.773816 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:25.773477 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:29:25.790885 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:25.790826 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" podStartSLOduration=2.067528297 podStartE2EDuration="4.790810013s" totalImagesPullingTime="2.723281716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:29:25.789758027 +0000 UTC m=+298.093926688" watchObservedRunningTime="2026-07-06 22:29:25.790810013 +0000 UTC m=+298.094978686" Jul 06 22:29:28.068086 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:28.068058 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:29:28.075148 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:28.075120 2572 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 22:29:28.078122 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:28.078093 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:29:56.780415 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:29:56.780376 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-h6l2m" Jul 06 22:30:50.065846 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.065811 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5dd4b778fb-5x8c6"] Jul 06 22:30:50.069400 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.069381 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.094639 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.094611 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dd4b778fb-5x8c6"] Jul 06 22:30:50.172346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172311 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-trusted-ca-bundle\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.172346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172354 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-oauth-serving-cert\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.172568 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172369 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdp7h\" (UniqueName: \"kubernetes.io/projected/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-kube-api-access-wdp7h\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.172568 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172448 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-service-ca\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.172568 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172467 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-oauth-config\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.172568 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172485 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-config\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.172568 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.172511 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-serving-cert\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273048 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273019 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-oauth-config\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273051 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-config\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273081 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-serving-cert\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273120 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-trusted-ca-bundle\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273145 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-oauth-serving-cert\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273206 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273161 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdp7h\" (UniqueName: \"kubernetes.io/projected/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-kube-api-access-wdp7h\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273475 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-service-ca\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.273969 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273945 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-service-ca\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.274074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.273950 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-trusted-ca-bundle\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.274074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.274043 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-config\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.274153 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.274110 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-oauth-serving-cert\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.275717 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.275692 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-serving-cert\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.275793 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.275702 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-console-oauth-config\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.285309 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.285284 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdp7h\" (UniqueName: \"kubernetes.io/projected/af8a0f70-3396-4dd8-a24d-67dc2e2e6039-kube-api-access-wdp7h\") pod \"console-5dd4b778fb-5x8c6\" (UID: \"af8a0f70-3396-4dd8-a24d-67dc2e2e6039\") " pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.377500 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.377417 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:30:50.546084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:50.546060 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5dd4b778fb-5x8c6"] Jul 06 22:30:50.548309 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:30:50.548284 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8a0f70_3396_4dd8_a24d_67dc2e2e6039.slice/crio-87b2e73681a2bb9090b7e12a177d13dd8ed58fe031aff207ca962dfecd50ad03 WatchSource:0}: Error finding container 87b2e73681a2bb9090b7e12a177d13dd8ed58fe031aff207ca962dfecd50ad03: Status 404 returned error can't find the container with id 87b2e73681a2bb9090b7e12a177d13dd8ed58fe031aff207ca962dfecd50ad03 Jul 06 22:30:51.195575 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:51.195534 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dd4b778fb-5x8c6" event={"ID":"af8a0f70-3396-4dd8-a24d-67dc2e2e6039","Type":"ContainerStarted","Data":"e5edaf50558611c4f501f14b0e283951028b5f6369fa016c3bb6d0b89b9dc850"} Jul 06 22:30:51.195971 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:51.195598 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5dd4b778fb-5x8c6" event={"ID":"af8a0f70-3396-4dd8-a24d-67dc2e2e6039","Type":"ContainerStarted","Data":"87b2e73681a2bb9090b7e12a177d13dd8ed58fe031aff207ca962dfecd50ad03"} Jul 06 22:30:51.219181 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:51.219134 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5dd4b778fb-5x8c6" podStartSLOduration=1.219119298 podStartE2EDuration="1.219119298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:51.218351092 +0000 UTC m=+383.522519752" watchObservedRunningTime="2026-07-06 22:30:51.219119298 +0000 UTC m=+383.523288023" Jul 06 22:30:52.838776 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.838742 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm"] Jul 06 22:30:52.842226 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.842210 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.845150 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.845129 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 22:30:52.845351 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.845337 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 22:30:52.845401 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.845378 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 22:30:52.846867 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.846779 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xfnb6\"" Jul 06 22:30:52.847049 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.847023 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 22:30:52.848951 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.848923 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm"] Jul 06 22:30:52.896593 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.896558 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.896693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.896640 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whg28\" (UniqueName: \"kubernetes.io/projected/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-kube-api-access-whg28\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.896693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.896661 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.997862 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.997836 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whg28\" (UniqueName: \"kubernetes.io/projected/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-kube-api-access-whg28\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.998012 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.997871 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.998012 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.997913 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:52.998570 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:52.998553 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:53.000457 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:53.000437 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:53.011200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:53.011173 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whg28\" (UniqueName: \"kubernetes.io/projected/3b7d760d-80cc-4f6f-b37f-163db5b39c4e-kube-api-access-whg28\") pod \"kuadrant-console-plugin-66b5774985-j4shm\" (UID: \"3b7d760d-80cc-4f6f-b37f-163db5b39c4e\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:53.153423 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:53.153346 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" Jul 06 22:30:53.283465 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:53.283440 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm"] Jul 06 22:30:53.285282 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:30:53.285254 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b7d760d_80cc_4f6f_b37f_163db5b39c4e.slice/crio-39c15249896c2e6fd6a9c7974548e461ae370c4792428be6efeef91689e191a7 WatchSource:0}: Error finding container 39c15249896c2e6fd6a9c7974548e461ae370c4792428be6efeef91689e191a7: Status 404 returned error can't find the container with id 39c15249896c2e6fd6a9c7974548e461ae370c4792428be6efeef91689e191a7 Jul 06 22:30:53.286753 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:53.286733 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:30:54.215012 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:30:54.214973 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" event={"ID":"3b7d760d-80cc-4f6f-b37f-163db5b39c4e","Type":"ContainerStarted","Data":"39c15249896c2e6fd6a9c7974548e461ae370c4792428be6efeef91689e191a7"} Jul 06 22:31:00.377696 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:00.377656 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:31:00.384319 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:00.378009 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:31:00.390128 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:00.390108 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:31:01.257601 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:01.257575 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5dd4b778fb-5x8c6" Jul 06 22:31:01.307840 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:01.307724 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-588644757d-5ztqh"] Jul 06 22:31:16.336993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:16.336956 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" event={"ID":"3b7d760d-80cc-4f6f-b37f-163db5b39c4e","Type":"ContainerStarted","Data":"385660bfc828cad280cfa3e3c459dbb8d1b70feca92a05f40c500b5449669f15"} Jul 06 22:31:16.376058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:16.376004 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-j4shm" podStartSLOduration=2.114011345 podStartE2EDuration="24.37598873s" totalImagesPullingTime="22.261977385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:31:16.374786645 +0000 UTC m=+408.678955306" watchObservedRunningTime="2026-07-06 22:31:16.37598873 +0000 UTC m=+408.680157390" Jul 06 22:31:26.334055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.334013 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-588644757d-5ztqh" podUID="f36eba92-e5f8-486d-ae4b-6d7d7f288da4" containerName="console" containerID="cri-o://08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00" gracePeriod=15 Jul 06 22:31:26.585993 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.585936 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:31:26.728376 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728339 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-trusted-ca-bundle\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728537 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728411 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-oauth-serving-cert\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728537 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728482 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-serving-cert\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728537 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728520 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-service-ca\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728670 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728546 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lslk6\" (UniqueName: \"kubernetes.io/projected/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-kube-api-access-lslk6\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728670 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728606 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-config\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728670 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728647 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-oauth-config\") pod \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\" (UID: \"f36eba92-e5f8-486d-ae4b-6d7d7f288da4\") " Jul 06 22:31:26.728813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728791 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:31:26.728951 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728927 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:31:26.728951 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728942 2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-trusted-ca-bundle\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:26.729044 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.728965 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-service-ca" (OuterVolumeSpecName: "service-ca") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:31:26.729133 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.729108 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-config" (OuterVolumeSpecName: "console-config") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:31:26.730864 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.730836 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:31:26.730961 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.730893 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-kube-api-access-lslk6" (OuterVolumeSpecName: "kube-api-access-lslk6") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "kube-api-access-lslk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:31:26.730961 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.730910 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f36eba92-e5f8-486d-ae4b-6d7d7f288da4" (UID: "f36eba92-e5f8-486d-ae4b-6d7d7f288da4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:31:26.830390 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.830362 2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:26.830390 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.830387 2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-oauth-config\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:26.830390 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.830397 2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-oauth-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:26.830604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.830406 2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-console-serving-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:26.830604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.830414 2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:26.830604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:26.830422 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lslk6\" (UniqueName: \"kubernetes.io/projected/f36eba92-e5f8-486d-ae4b-6d7d7f288da4-kube-api-access-lslk6\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:27.388578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.388540 2572 generic.go:356] "Generic (PLEG): container finished" podID="f36eba92-e5f8-486d-ae4b-6d7d7f288da4" containerID="08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00" exitCode=0 Jul 06 22:31:27.389059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.388599 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-588644757d-5ztqh" Jul 06 22:31:27.389059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.388623 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588644757d-5ztqh" event={"ID":"f36eba92-e5f8-486d-ae4b-6d7d7f288da4","Type":"ContainerDied","Data":"08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00"} Jul 06 22:31:27.389059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.388671 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-588644757d-5ztqh" event={"ID":"f36eba92-e5f8-486d-ae4b-6d7d7f288da4","Type":"ContainerDied","Data":"dfa8c2e41f7970a7bdbefcaf7c16512d457305d0f277a85cfd411030b87b5118"} Jul 06 22:31:27.389059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.388691 2572 scope.go:117] "RemoveContainer" containerID="08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00" Jul 06 22:31:27.397761 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.397747 2572 scope.go:117] "RemoveContainer" containerID="08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00" Jul 06 22:31:27.397983 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:31:27.397967 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00\": container with ID starting with 08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00 not found: ID does not exist" containerID="08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00" Jul 06 22:31:27.398023 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.397991 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00"} err="failed to get container status \"08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00\": rpc error: code = NotFound desc = could not find container \"08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00\": container with ID starting with 08c7bcd88e4360f5b2d358c8a6ce59f45cb366c38afb1545d1830b669ae75c00 not found: ID does not exist" Jul 06 22:31:27.411676 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.411648 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-588644757d-5ztqh"] Jul 06 22:31:27.418542 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:27.418520 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-588644757d-5ztqh"] Jul 06 22:31:28.248110 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:28.248081 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f36eba92-e5f8-486d-ae4b-6d7d7f288da4" path="/var/lib/kubelet/pods/f36eba92-e5f8-486d-ae4b-6d7d7f288da4/volumes" Jul 06 22:31:35.180028 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.179998 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:35.180480 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.180465 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f36eba92-e5f8-486d-ae4b-6d7d7f288da4" containerName="console" Jul 06 22:31:35.180533 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.180482 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f36eba92-e5f8-486d-ae4b-6d7d7f288da4" containerName="console" Jul 06 22:31:35.180569 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.180540 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f36eba92-e5f8-486d-ae4b-6d7d7f288da4" containerName="console" Jul 06 22:31:35.183508 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.183491 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.187794 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.187778 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 22:31:35.205659 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.205620 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:35.266658 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.266620 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:35.306423 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.306392 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9xqs\" (UniqueName: \"kubernetes.io/projected/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-kube-api-access-m9xqs\") pod \"limitador-limitador-64c94fd64f-x7dzr\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.306539 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.306524 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-config-file\") pod \"limitador-limitador-64c94fd64f-x7dzr\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.407641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.407610 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-config-file\") pod \"limitador-limitador-64c94fd64f-x7dzr\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.407789 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.407650 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9xqs\" (UniqueName: \"kubernetes.io/projected/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-kube-api-access-m9xqs\") pod \"limitador-limitador-64c94fd64f-x7dzr\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.408322 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.408303 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-config-file\") pod \"limitador-limitador-64c94fd64f-x7dzr\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.418662 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.418635 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9xqs\" (UniqueName: \"kubernetes.io/projected/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-kube-api-access-m9xqs\") pod \"limitador-limitador-64c94fd64f-x7dzr\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.494655 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.494572 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:35.664242 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:35.664209 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:35.666078 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:31:35.666048 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb59f6df6_a682_43d9_bf7c_4b6bb616d1d0.slice/crio-8302078c6689f12ecff4cbb0b08ae23fb09d892b7ee2c4c36ea8b26b4aceb16b WatchSource:0}: Error finding container 8302078c6689f12ecff4cbb0b08ae23fb09d892b7ee2c4c36ea8b26b4aceb16b: Status 404 returned error can't find the container with id 8302078c6689f12ecff4cbb0b08ae23fb09d892b7ee2c4c36ea8b26b4aceb16b Jul 06 22:31:36.431191 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:36.431146 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" event={"ID":"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0","Type":"ContainerStarted","Data":"8302078c6689f12ecff4cbb0b08ae23fb09d892b7ee2c4c36ea8b26b4aceb16b"} Jul 06 22:31:39.447458 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:39.447419 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" event={"ID":"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0","Type":"ContainerStarted","Data":"b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528"} Jul 06 22:31:39.447912 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:39.447497 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:50.214216 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.214114 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" podStartSLOduration=12.070852841 podStartE2EDuration="15.214097483s" totalImagesPullingTime="3.143244642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:31:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:31:39.472892775 +0000 UTC m=+431.777061435" watchObservedRunningTime="2026-07-06 22:31:50.214097483 +0000 UTC m=+442.518266153" Jul 06 22:31:50.214596 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.214454 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:50.214659 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.214638 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" podUID="b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" containerName="limitador" containerID="cri-o://b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528" gracePeriod=30 Jul 06 22:31:50.215398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.215362 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:50.765299 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.765277 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:50.844406 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.844328 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9xqs\" (UniqueName: \"kubernetes.io/projected/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-kube-api-access-m9xqs\") pod \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " Jul 06 22:31:50.844406 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.844372 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-config-file\") pod \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\" (UID: \"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0\") " Jul 06 22:31:50.844772 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.844742 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-config-file" (OuterVolumeSpecName: "config-file") pod "b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" (UID: "b59f6df6-a682-43d9-bf7c-4b6bb616d1d0"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:31:50.846641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.846620 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-kube-api-access-m9xqs" (OuterVolumeSpecName: "kube-api-access-m9xqs") pod "b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" (UID: "b59f6df6-a682-43d9-bf7c-4b6bb616d1d0"). InnerVolumeSpecName "kube-api-access-m9xqs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:31:50.945228 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.945198 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9xqs\" (UniqueName: \"kubernetes.io/projected/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-kube-api-access-m9xqs\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:50.945228 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:50.945222 2572 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0-config-file\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:31:51.502821 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.502789 2572 generic.go:356] "Generic (PLEG): container finished" podID="b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" containerID="b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528" exitCode=0 Jul 06 22:31:51.503200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.502833 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" Jul 06 22:31:51.503200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.502874 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" event={"ID":"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0","Type":"ContainerDied","Data":"b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528"} Jul 06 22:31:51.503200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.502929 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-x7dzr" event={"ID":"b59f6df6-a682-43d9-bf7c-4b6bb616d1d0","Type":"ContainerDied","Data":"8302078c6689f12ecff4cbb0b08ae23fb09d892b7ee2c4c36ea8b26b4aceb16b"} Jul 06 22:31:51.503200 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.502952 2572 scope.go:117] "RemoveContainer" containerID="b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528" Jul 06 22:31:51.511842 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.511827 2572 scope.go:117] "RemoveContainer" containerID="b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528" Jul 06 22:31:51.512083 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:31:51.512065 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528\": container with ID starting with b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528 not found: ID does not exist" containerID="b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528" Jul 06 22:31:51.512132 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.512091 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528"} err="failed to get container status \"b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528\": rpc error: code = NotFound desc = could not find container \"b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528\": container with ID starting with b7c9eb39a897b76545a46b3a71ce0768c580e4e257221f4745ad20280594f528 not found: ID does not exist" Jul 06 22:31:51.524704 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.524682 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:51.528664 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:51.528641 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-x7dzr"] Jul 06 22:31:52.247878 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:52.247845 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" path="/var/lib/kubelet/pods/b59f6df6-a682-43d9-bf7c-4b6bb616d1d0/volumes" Jul 06 22:31:56.636676 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.636641 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-bz7wl"] Jul 06 22:31:56.637056 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.636988 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" containerName="limitador" Jul 06 22:31:56.637056 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.637000 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" containerName="limitador" Jul 06 22:31:56.637131 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.637068 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="b59f6df6-a682-43d9-bf7c-4b6bb616d1d0" containerName="limitador" Jul 06 22:31:56.639644 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.639624 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.644301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.644277 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-gfncp\"" Jul 06 22:31:56.644440 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.644374 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 22:31:56.667885 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.667856 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-bz7wl"] Jul 06 22:31:56.700762 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.700732 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/94ca973b-3a14-47ba-b1ee-c5ab3dfadb06-data\") pod \"postgres-69fb4594cf-bz7wl\" (UID: \"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06\") " pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.700945 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.700786 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkp78\" (UniqueName: \"kubernetes.io/projected/94ca973b-3a14-47ba-b1ee-c5ab3dfadb06-kube-api-access-lkp78\") pod \"postgres-69fb4594cf-bz7wl\" (UID: \"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06\") " pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.801991 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.801959 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/94ca973b-3a14-47ba-b1ee-c5ab3dfadb06-data\") pod \"postgres-69fb4594cf-bz7wl\" (UID: \"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06\") " pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.802146 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.802002 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkp78\" (UniqueName: \"kubernetes.io/projected/94ca973b-3a14-47ba-b1ee-c5ab3dfadb06-kube-api-access-lkp78\") pod \"postgres-69fb4594cf-bz7wl\" (UID: \"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06\") " pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.802337 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.802317 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/94ca973b-3a14-47ba-b1ee-c5ab3dfadb06-data\") pod \"postgres-69fb4594cf-bz7wl\" (UID: \"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06\") " pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.812181 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.812153 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkp78\" (UniqueName: \"kubernetes.io/projected/94ca973b-3a14-47ba-b1ee-c5ab3dfadb06-kube-api-access-lkp78\") pod \"postgres-69fb4594cf-bz7wl\" (UID: \"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06\") " pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:56.949833 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:56.949803 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:31:57.087183 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:57.087158 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-bz7wl"] Jul 06 22:31:57.089896 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:31:57.089871 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94ca973b_3a14_47ba_b1ee_c5ab3dfadb06.slice/crio-561c7f12392b3374619d768b4d7a6301dfa183eefa1c122b37871a49dd9188b6 WatchSource:0}: Error finding container 561c7f12392b3374619d768b4d7a6301dfa183eefa1c122b37871a49dd9188b6: Status 404 returned error can't find the container with id 561c7f12392b3374619d768b4d7a6301dfa183eefa1c122b37871a49dd9188b6 Jul 06 22:31:57.532477 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:31:57.532445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-bz7wl" event={"ID":"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06","Type":"ContainerStarted","Data":"561c7f12392b3374619d768b4d7a6301dfa183eefa1c122b37871a49dd9188b6"} Jul 06 22:32:02.561606 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:02.561557 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-bz7wl" event={"ID":"94ca973b-3a14-47ba-b1ee-c5ab3dfadb06","Type":"ContainerStarted","Data":"54f3f6d43915b7f06d78cf6ccbb8f9c991a4e7c71b657ffdaf30bc2ce6a45e78"} Jul 06 22:32:02.562012 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:02.561629 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:32:02.592037 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:02.591981 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-bz7wl" podStartSLOduration=1.839497285 podStartE2EDuration="6.591964195s" totalImagesPullingTime="4.75246691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:02.589099175 +0000 UTC m=+454.893267833" watchObservedRunningTime="2026-07-06 22:32:02.591964195 +0000 UTC m=+454.896132866" Jul 06 22:32:08.596910 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:08.596875 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-bz7wl" Jul 06 22:32:15.512525 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.512492 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-f9b9c4bcd-gzrp7"] Jul 06 22:32:15.534970 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.534941 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9b9c4bcd-gzrp7"] Jul 06 22:32:15.535153 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.535045 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.537956 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.537935 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 22:32:15.538085 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.537976 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 22:32:15.538184 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.538167 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-f4vpz\"" Jul 06 22:32:15.585450 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.585421 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d54b01c-c0a7-401f-8416-6b19415a81d4-webhook-cert\") pod \"maas-controller-f9b9c4bcd-gzrp7\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.585616 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.585457 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cndtn\" (UniqueName: \"kubernetes.io/projected/6d54b01c-c0a7-401f-8416-6b19415a81d4-kube-api-access-cndtn\") pod \"maas-controller-f9b9c4bcd-gzrp7\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.686854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.686823 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d54b01c-c0a7-401f-8416-6b19415a81d4-webhook-cert\") pod \"maas-controller-f9b9c4bcd-gzrp7\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.686854 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.686864 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cndtn\" (UniqueName: \"kubernetes.io/projected/6d54b01c-c0a7-401f-8416-6b19415a81d4-kube-api-access-cndtn\") pod \"maas-controller-f9b9c4bcd-gzrp7\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.689588 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.689519 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d54b01c-c0a7-401f-8416-6b19415a81d4-webhook-cert\") pod \"maas-controller-f9b9c4bcd-gzrp7\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.692070 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.692039 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d57c96767-khxnr"] Jul 06 22:32:15.695604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.695571 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.700012 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.699984 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cndtn\" (UniqueName: \"kubernetes.io/projected/6d54b01c-c0a7-401f-8416-6b19415a81d4-kube-api-access-cndtn\") pod \"maas-controller-f9b9c4bcd-gzrp7\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.707159 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.707133 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d57c96767-khxnr"] Jul 06 22:32:15.788154 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.788069 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqc5b\" (UniqueName: \"kubernetes.io/projected/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-kube-api-access-vqc5b\") pod \"maas-controller-d57c96767-khxnr\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.788327 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.788174 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-webhook-cert\") pod \"maas-controller-d57c96767-khxnr\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.846736 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.846701 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:15.889114 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.889069 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-webhook-cert\") pod \"maas-controller-d57c96767-khxnr\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.889295 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.889149 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqc5b\" (UniqueName: \"kubernetes.io/projected/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-kube-api-access-vqc5b\") pod \"maas-controller-d57c96767-khxnr\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.891709 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.891684 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-webhook-cert\") pod \"maas-controller-d57c96767-khxnr\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.903481 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.903454 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqc5b\" (UniqueName: \"kubernetes.io/projected/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-kube-api-access-vqc5b\") pod \"maas-controller-d57c96767-khxnr\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:15.989551 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:15.989508 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-f9b9c4bcd-gzrp7"] Jul 06 22:32:15.992348 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:32:15.992306 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d54b01c_c0a7_401f_8416_6b19415a81d4.slice/crio-d95f6da4d28804c5d7258d0bf57b284ec5ed7a79761a214011c43e9401fdd811 WatchSource:0}: Error finding container d95f6da4d28804c5d7258d0bf57b284ec5ed7a79761a214011c43e9401fdd811: Status 404 returned error can't find the container with id d95f6da4d28804c5d7258d0bf57b284ec5ed7a79761a214011c43e9401fdd811 Jul 06 22:32:16.018956 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:16.018927 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:16.154133 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:16.154108 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d57c96767-khxnr"] Jul 06 22:32:16.155946 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:32:16.155913 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5059c1fe_c1f2_4ce2_a0cb_a6634858ccad.slice/crio-33179580e8d4537ce2a95e0e90ed50df9e2f569e4fee005ca1b62186bdc48cd7 WatchSource:0}: Error finding container 33179580e8d4537ce2a95e0e90ed50df9e2f569e4fee005ca1b62186bdc48cd7: Status 404 returned error can't find the container with id 33179580e8d4537ce2a95e0e90ed50df9e2f569e4fee005ca1b62186bdc48cd7 Jul 06 22:32:16.629876 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:16.629837 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d57c96767-khxnr" event={"ID":"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad","Type":"ContainerStarted","Data":"33179580e8d4537ce2a95e0e90ed50df9e2f569e4fee005ca1b62186bdc48cd7"} Jul 06 22:32:16.631394 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:16.631366 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" event={"ID":"6d54b01c-c0a7-401f-8416-6b19415a81d4","Type":"ContainerStarted","Data":"d95f6da4d28804c5d7258d0bf57b284ec5ed7a79761a214011c43e9401fdd811"} Jul 06 22:32:19.649811 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:19.649769 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" event={"ID":"6d54b01c-c0a7-401f-8416-6b19415a81d4","Type":"ContainerStarted","Data":"1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955"} Jul 06 22:32:19.650208 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:19.649846 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:19.651170 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:19.651151 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d57c96767-khxnr" event={"ID":"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad","Type":"ContainerStarted","Data":"07dbadffa6f767a4fdae84d2b1fcf755cc673058f9d8611ce410982ab0a2e534"} Jul 06 22:32:19.651264 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:19.651258 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:19.671466 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:19.671420 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" podStartSLOduration=1.914855619 podStartE2EDuration="4.671407031s" totalImagesPullingTime="2.756551412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:19.669828323 +0000 UTC m=+471.973996981" watchObservedRunningTime="2026-07-06 22:32:19.671407031 +0000 UTC m=+471.975575688" Jul 06 22:32:19.694080 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:19.694037 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d57c96767-khxnr" podStartSLOduration=2.112278385 podStartE2EDuration="4.694023658s" totalImagesPullingTime="2.581745273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:19.693723922 +0000 UTC m=+471.997892579" watchObservedRunningTime="2026-07-06 22:32:19.694023658 +0000 UTC m=+471.998192349" Jul 06 22:32:30.661155 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:30.661117 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:32:30.661680 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:30.661200 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:30.722608 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:30.722575 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f9b9c4bcd-gzrp7"] Jul 06 22:32:30.722813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:30.722787 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" podUID="6d54b01c-c0a7-401f-8416-6b19415a81d4" containerName="manager" containerID="cri-o://1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955" gracePeriod=30 Jul 06 22:32:30.972514 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:30.972491 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:31.020072 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.020038 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d54b01c-c0a7-401f-8416-6b19415a81d4-webhook-cert\") pod \"6d54b01c-c0a7-401f-8416-6b19415a81d4\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " Jul 06 22:32:31.020254 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.020133 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cndtn\" (UniqueName: \"kubernetes.io/projected/6d54b01c-c0a7-401f-8416-6b19415a81d4-kube-api-access-cndtn\") pod \"6d54b01c-c0a7-401f-8416-6b19415a81d4\" (UID: \"6d54b01c-c0a7-401f-8416-6b19415a81d4\") " Jul 06 22:32:31.022443 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.022412 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d54b01c-c0a7-401f-8416-6b19415a81d4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6d54b01c-c0a7-401f-8416-6b19415a81d4" (UID: "6d54b01c-c0a7-401f-8416-6b19415a81d4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:32:31.022443 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.022424 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d54b01c-c0a7-401f-8416-6b19415a81d4-kube-api-access-cndtn" (OuterVolumeSpecName: "kube-api-access-cndtn") pod "6d54b01c-c0a7-401f-8416-6b19415a81d4" (UID: "6d54b01c-c0a7-401f-8416-6b19415a81d4"). InnerVolumeSpecName "kube-api-access-cndtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:32:31.121255 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.121193 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d54b01c-c0a7-401f-8416-6b19415a81d4-webhook-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:32:31.121255 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.121227 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cndtn\" (UniqueName: \"kubernetes.io/projected/6d54b01c-c0a7-401f-8416-6b19415a81d4-kube-api-access-cndtn\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:32:31.707918 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.707879 2572 generic.go:356] "Generic (PLEG): container finished" podID="6d54b01c-c0a7-401f-8416-6b19415a81d4" containerID="1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955" exitCode=0 Jul 06 22:32:31.707918 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.707922 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" Jul 06 22:32:31.708496 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.707930 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" event={"ID":"6d54b01c-c0a7-401f-8416-6b19415a81d4","Type":"ContainerDied","Data":"1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955"} Jul 06 22:32:31.708496 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.707983 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-f9b9c4bcd-gzrp7" event={"ID":"6d54b01c-c0a7-401f-8416-6b19415a81d4","Type":"ContainerDied","Data":"d95f6da4d28804c5d7258d0bf57b284ec5ed7a79761a214011c43e9401fdd811"} Jul 06 22:32:31.708496 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.708012 2572 scope.go:117] "RemoveContainer" containerID="1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955" Jul 06 22:32:31.716890 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.716873 2572 scope.go:117] "RemoveContainer" containerID="1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955" Jul 06 22:32:31.717149 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:32:31.717119 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955\": container with ID starting with 1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955 not found: ID does not exist" containerID="1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955" Jul 06 22:32:31.717187 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.717159 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955"} err="failed to get container status \"1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955\": rpc error: code = NotFound desc = could not find container \"1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955\": container with ID starting with 1b37cdbb59594a5ef7c20d6a49c5e6970fd56a8ee4311ba1e3fcce160ac30955 not found: ID does not exist" Jul 06 22:32:31.730018 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.729996 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-f9b9c4bcd-gzrp7"] Jul 06 22:32:31.735835 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:31.735813 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-f9b9c4bcd-gzrp7"] Jul 06 22:32:32.247779 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:32.247743 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d54b01c-c0a7-401f-8416-6b19415a81d4" path="/var/lib/kubelet/pods/6d54b01c-c0a7-401f-8416-6b19415a81d4/volumes" Jul 06 22:32:52.004633 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.004595 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-69b48b77f8-dvmfq"] Jul 06 22:32:52.005208 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.005171 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d54b01c-c0a7-401f-8416-6b19415a81d4" containerName="manager" Jul 06 22:32:52.005208 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.005192 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d54b01c-c0a7-401f-8416-6b19415a81d4" containerName="manager" Jul 06 22:32:52.005358 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.005313 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d54b01c-c0a7-401f-8416-6b19415a81d4" containerName="manager" Jul 06 22:32:52.010870 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.010845 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.013821 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.013798 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 22:32:52.013959 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.013826 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-f95qn\"" Jul 06 22:32:52.020880 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.020854 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-69b48b77f8-dvmfq"] Jul 06 22:32:52.121029 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.120986 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j2xn\" (UniqueName: \"kubernetes.io/projected/d46a7afc-865a-4459-9ae3-aaf250fc623d-kube-api-access-9j2xn\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.121226 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.121070 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d46a7afc-865a-4459-9ae3-aaf250fc623d-maas-api-tls\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.121226 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.121117 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d46a7afc-865a-4459-9ae3-aaf250fc623d-openshift-service-ca\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.221916 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.221882 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d46a7afc-865a-4459-9ae3-aaf250fc623d-maas-api-tls\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.222091 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.221931 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d46a7afc-865a-4459-9ae3-aaf250fc623d-openshift-service-ca\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.222091 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.221982 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j2xn\" (UniqueName: \"kubernetes.io/projected/d46a7afc-865a-4459-9ae3-aaf250fc623d-kube-api-access-9j2xn\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.222091 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:32:52.222036 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 06 22:32:52.222225 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:32:52.222108 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d46a7afc-865a-4459-9ae3-aaf250fc623d-maas-api-tls podName:d46a7afc-865a-4459-9ae3-aaf250fc623d nodeName:}" failed. No retries permitted until 2026-07-06 22:32:52.722091382 +0000 UTC m=+505.026260018 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d46a7afc-865a-4459-9ae3-aaf250fc623d-maas-api-tls") pod "maas-api-69b48b77f8-dvmfq" (UID: "d46a7afc-865a-4459-9ae3-aaf250fc623d") : secret "maas-api-serving-cert" not found Jul 06 22:32:52.222658 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.222627 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d46a7afc-865a-4459-9ae3-aaf250fc623d-openshift-service-ca\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.232519 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.232491 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j2xn\" (UniqueName: \"kubernetes.io/projected/d46a7afc-865a-4459-9ae3-aaf250fc623d-kube-api-access-9j2xn\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.442688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.442639 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-km4lp"] Jul 06 22:32:52.481034 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.480997 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-km4lp"] Jul 06 22:32:52.481199 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.481047 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-88j4z"] Jul 06 22:32:52.481199 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.481135 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.484112 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.484088 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 22:32:52.512000 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.511966 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-88j4z"] Jul 06 22:32:52.512165 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.512062 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.516424 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.516392 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-4hccr\"" Jul 06 22:32:52.626366 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.626331 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54gk9\" (UniqueName: \"kubernetes.io/projected/092bfe7a-d748-44dc-80d2-31d5d215d726-kube-api-access-54gk9\") pod \"payload-processing-58f86c7798-88j4z\" (UID: \"092bfe7a-d748-44dc-80d2-31d5d215d726\") " pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.626538 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.626382 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434b4986-8680-4a50-b734-dad514382810-plugins-config-volume\") pod \"payload-pre-processing-7964457696-km4lp\" (UID: \"434b4986-8680-4a50-b734-dad514382810\") " pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.626580 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.626537 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv2bq\" (UniqueName: \"kubernetes.io/projected/434b4986-8680-4a50-b734-dad514382810-kube-api-access-qv2bq\") pod \"payload-pre-processing-7964457696-km4lp\" (UID: \"434b4986-8680-4a50-b734-dad514382810\") " pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.626614 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.626576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092bfe7a-d748-44dc-80d2-31d5d215d726-plugins-config-volume\") pod \"payload-processing-58f86c7798-88j4z\" (UID: \"092bfe7a-d748-44dc-80d2-31d5d215d726\") " pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.728153 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728062 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv2bq\" (UniqueName: \"kubernetes.io/projected/434b4986-8680-4a50-b734-dad514382810-kube-api-access-qv2bq\") pod \"payload-pre-processing-7964457696-km4lp\" (UID: \"434b4986-8680-4a50-b734-dad514382810\") " pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.728153 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728115 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092bfe7a-d748-44dc-80d2-31d5d215d726-plugins-config-volume\") pod \"payload-processing-58f86c7798-88j4z\" (UID: \"092bfe7a-d748-44dc-80d2-31d5d215d726\") " pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.728389 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728160 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d46a7afc-865a-4459-9ae3-aaf250fc623d-maas-api-tls\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.728389 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728203 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54gk9\" (UniqueName: \"kubernetes.io/projected/092bfe7a-d748-44dc-80d2-31d5d215d726-kube-api-access-54gk9\") pod \"payload-processing-58f86c7798-88j4z\" (UID: \"092bfe7a-d748-44dc-80d2-31d5d215d726\") " pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.728389 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728268 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434b4986-8680-4a50-b734-dad514382810-plugins-config-volume\") pod \"payload-pre-processing-7964457696-km4lp\" (UID: \"434b4986-8680-4a50-b734-dad514382810\") " pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.728947 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728922 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/434b4986-8680-4a50-b734-dad514382810-plugins-config-volume\") pod \"payload-pre-processing-7964457696-km4lp\" (UID: \"434b4986-8680-4a50-b734-dad514382810\") " pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.729061 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.728996 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/092bfe7a-d748-44dc-80d2-31d5d215d726-plugins-config-volume\") pod \"payload-processing-58f86c7798-88j4z\" (UID: \"092bfe7a-d748-44dc-80d2-31d5d215d726\") " pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.730817 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.730787 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d46a7afc-865a-4459-9ae3-aaf250fc623d-maas-api-tls\") pod \"maas-api-69b48b77f8-dvmfq\" (UID: \"d46a7afc-865a-4459-9ae3-aaf250fc623d\") " pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.736253 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.736216 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv2bq\" (UniqueName: \"kubernetes.io/projected/434b4986-8680-4a50-b734-dad514382810-kube-api-access-qv2bq\") pod \"payload-pre-processing-7964457696-km4lp\" (UID: \"434b4986-8680-4a50-b734-dad514382810\") " pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.736362 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.736293 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54gk9\" (UniqueName: \"kubernetes.io/projected/092bfe7a-d748-44dc-80d2-31d5d215d726-kube-api-access-54gk9\") pod \"payload-processing-58f86c7798-88j4z\" (UID: \"092bfe7a-d748-44dc-80d2-31d5d215d726\") " pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.793228 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.793192 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:52.822639 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.822610 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:52.922635 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.922604 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:52.974571 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:52.974527 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-km4lp"] Jul 06 22:32:52.977651 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:32:52.977605 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434b4986_8680_4a50_b734_dad514382810.slice/crio-4f32695c586e74047e41a0208e4a6f20f5f4587b2ce42062a00c6a5852f9bc43 WatchSource:0}: Error finding container 4f32695c586e74047e41a0208e4a6f20f5f4587b2ce42062a00c6a5852f9bc43: Status 404 returned error can't find the container with id 4f32695c586e74047e41a0208e4a6f20f5f4587b2ce42062a00c6a5852f9bc43 Jul 06 22:32:53.004380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:53.004123 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-88j4z"] Jul 06 22:32:53.006829 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:32:53.006798 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod092bfe7a_d748_44dc_80d2_31d5d215d726.slice/crio-c30f9cbf548b4523f4aea2edc747ed380e98603c5664fc9218db191594e2dcc0 WatchSource:0}: Error finding container c30f9cbf548b4523f4aea2edc747ed380e98603c5664fc9218db191594e2dcc0: Status 404 returned error can't find the container with id c30f9cbf548b4523f4aea2edc747ed380e98603c5664fc9218db191594e2dcc0 Jul 06 22:32:53.075912 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:53.075887 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-69b48b77f8-dvmfq"] Jul 06 22:32:53.077864 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:32:53.077830 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd46a7afc_865a_4459_9ae3_aaf250fc623d.slice/crio-9af160ab3e9887fe066be9cd1ab4d9de3ee527c0c68660a7aee38c15a6f40b67 WatchSource:0}: Error finding container 9af160ab3e9887fe066be9cd1ab4d9de3ee527c0c68660a7aee38c15a6f40b67: Status 404 returned error can't find the container with id 9af160ab3e9887fe066be9cd1ab4d9de3ee527c0c68660a7aee38c15a6f40b67 Jul 06 22:32:53.821084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:53.821041 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" event={"ID":"434b4986-8680-4a50-b734-dad514382810","Type":"ContainerStarted","Data":"4f32695c586e74047e41a0208e4a6f20f5f4587b2ce42062a00c6a5852f9bc43"} Jul 06 22:32:53.822419 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:53.822387 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" event={"ID":"092bfe7a-d748-44dc-80d2-31d5d215d726","Type":"ContainerStarted","Data":"c30f9cbf548b4523f4aea2edc747ed380e98603c5664fc9218db191594e2dcc0"} Jul 06 22:32:53.824089 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:53.824061 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-69b48b77f8-dvmfq" event={"ID":"d46a7afc-865a-4459-9ae3-aaf250fc623d","Type":"ContainerStarted","Data":"9af160ab3e9887fe066be9cd1ab4d9de3ee527c0c68660a7aee38c15a6f40b67"} Jul 06 22:32:55.837380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.837335 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" event={"ID":"092bfe7a-d748-44dc-80d2-31d5d215d726","Type":"ContainerStarted","Data":"241f2efa4ac367fc4bd99350d91f5f09a340dd5f95a5f6a8284923afe2342291"} Jul 06 22:32:55.837808 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.837404 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:32:55.838825 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.838801 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-69b48b77f8-dvmfq" event={"ID":"d46a7afc-865a-4459-9ae3-aaf250fc623d","Type":"ContainerStarted","Data":"e6bb3b76f319e3065f70cd01a6fabbc3c998f39a1fbec1178325580dac9a9a31"} Jul 06 22:32:55.838930 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.838884 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:32:55.840263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.840222 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" event={"ID":"434b4986-8680-4a50-b734-dad514382810","Type":"ContainerStarted","Data":"0dc5a85cb1636118eea0d71a6fe9d58ca41771a1fbcb3798e663c1822f261366"} Jul 06 22:32:55.840383 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.840365 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:32:55.853621 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.853571 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" podStartSLOduration=2.050805146 podStartE2EDuration="3.853556531s" totalImagesPullingTime="1.802751385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:55.853178006 +0000 UTC m=+508.157346666" watchObservedRunningTime="2026-07-06 22:32:55.853556531 +0000 UTC m=+508.157725189" Jul 06 22:32:55.875255 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.871822 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-69b48b77f8-dvmfq" podStartSLOduration=3.142300925 podStartE2EDuration="4.871802157s" totalImagesPullingTime="1.729501232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:55.869258666 +0000 UTC m=+508.173427326" watchObservedRunningTime="2026-07-06 22:32:55.871802157 +0000 UTC m=+508.175970817" Jul 06 22:32:55.890472 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:32:55.890407 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" podStartSLOduration=2.061925965 podStartE2EDuration="3.890389049s" totalImagesPullingTime="1.828463084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:32:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:32:55.889674462 +0000 UTC m=+508.193843135" watchObservedRunningTime="2026-07-06 22:32:55.890389049 +0000 UTC m=+508.194557719" Jul 06 22:33:01.851077 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:01.851044 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-69b48b77f8-dvmfq" Jul 06 22:33:06.847833 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:06.847793 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-km4lp" Jul 06 22:33:06.848326 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:06.847888 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-88j4z" Jul 06 22:33:43.094301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.094270 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk"] Jul 06 22:33:43.097849 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.097832 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.100744 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.100721 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 22:33:43.100881 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.100721 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 22:33:43.102128 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.102108 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-w2bxs\"" Jul 06 22:33:43.102204 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.102121 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 22:33:43.107518 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.107495 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk"] Jul 06 22:33:43.190057 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.190025 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fh9bx\" (UniqueName: \"kubernetes.io/projected/8d692b86-229a-44ce-88e7-9b217716e128-kube-api-access-fh9bx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.190266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.190066 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d692b86-229a-44ce-88e7-9b217716e128-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.190266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.190107 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.190266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.190162 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.190266 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.190182 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.190430 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.190287 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291178 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291148 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291189 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291213 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291266 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291289 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fh9bx\" (UniqueName: \"kubernetes.io/projected/8d692b86-229a-44ce-88e7-9b217716e128-kube-api-access-fh9bx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291380 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291316 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d692b86-229a-44ce-88e7-9b217716e128-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291657 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291619 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291657 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291627 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.291760 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.291706 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.293687 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.293664 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d692b86-229a-44ce-88e7-9b217716e128-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.293834 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.293818 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d692b86-229a-44ce-88e7-9b217716e128-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.304837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.304814 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fh9bx\" (UniqueName: \"kubernetes.io/projected/8d692b86-229a-44ce-88e7-9b217716e128-kube-api-access-fh9bx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-mgqxk\" (UID: \"8d692b86-229a-44ce-88e7-9b217716e128\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.410082 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.410005 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:43.542770 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:43.542742 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk"] Jul 06 22:33:43.544983 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:33:43.544957 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d692b86_229a_44ce_88e7_9b217716e128.slice/crio-e3c8d7fcef23a1325dc11b35b5149ddc441d390033a5d0953572a280bdef7341 WatchSource:0}: Error finding container e3c8d7fcef23a1325dc11b35b5149ddc441d390033a5d0953572a280bdef7341: Status 404 returned error can't find the container with id e3c8d7fcef23a1325dc11b35b5149ddc441d390033a5d0953572a280bdef7341 Jul 06 22:33:44.089869 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:44.089829 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" event={"ID":"8d692b86-229a-44ce-88e7-9b217716e128","Type":"ContainerStarted","Data":"e3c8d7fcef23a1325dc11b35b5149ddc441d390033a5d0953572a280bdef7341"} Jul 06 22:33:45.098928 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:45.098894 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" event={"ID":"8d692b86-229a-44ce-88e7-9b217716e128","Type":"ContainerStarted","Data":"7afd70c3810c4353046f54ba29284478b2f8e63b1190f7b3bbe8bec4a280c45c"} Jul 06 22:33:45.117978 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:45.117930 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" podStartSLOduration=1.1082255619999999 podStartE2EDuration="2.117914913s" totalImagesPullingTime="1.009689351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:33:45.11586799 +0000 UTC m=+557.420036672" watchObservedRunningTime="2026-07-06 22:33:45.117914913 +0000 UTC m=+557.422083629" Jul 06 22:33:53.410431 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:53.410383 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:53.410801 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:53.410463 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:53.422839 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:53.422809 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:54.154837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:54.154811 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-mgqxk" Jul 06 22:33:58.002604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.002573 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7"] Jul 06 22:33:58.005735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.005712 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.008666 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.008643 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 06 22:33:58.017061 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.017039 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7"] Jul 06 22:33:58.023476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.023451 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.023591 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.023495 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.023591 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.023576 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.023674 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.023617 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.023674 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.023659 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.023823 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.023797 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf9cl\" (UniqueName: \"kubernetes.io/projected/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-kube-api-access-wf9cl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.124696 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.124664 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.124875 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.124709 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.124875 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.124745 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.124875 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.124794 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.124875 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.124853 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.125129 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.124898 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf9cl\" (UniqueName: \"kubernetes.io/projected/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-kube-api-access-wf9cl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.125220 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.125196 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.125310 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.125284 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.125310 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.125303 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.127149 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.127126 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.127344 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.127325 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.133159 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.133137 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf9cl\" (UniqueName: \"kubernetes.io/projected/d31d87b7-8bae-4f46-aab3-01dc25bf22e9-kube-api-access-wf9cl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dhhw7\" (UID: \"d31d87b7-8bae-4f46-aab3-01dc25bf22e9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.317101 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.317018 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:33:58.448700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:58.448676 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7"] Jul 06 22:33:58.450645 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:33:58.450609 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd31d87b7_8bae_4f46_aab3_01dc25bf22e9.slice/crio-b2c2117146ff1684c8c4a90799ac2c903e7c345e48977a5fe8676b3bb646e956 WatchSource:0}: Error finding container b2c2117146ff1684c8c4a90799ac2c903e7c345e48977a5fe8676b3bb646e956: Status 404 returned error can't find the container with id b2c2117146ff1684c8c4a90799ac2c903e7c345e48977a5fe8676b3bb646e956 Jul 06 22:33:59.169825 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:59.169785 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" event={"ID":"d31d87b7-8bae-4f46-aab3-01dc25bf22e9","Type":"ContainerStarted","Data":"c793fdf67d1eb841b96ff6d0de58fef2c04b3da52042f248bc56e88b22679478"} Jul 06 22:33:59.170173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:59.169847 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" event={"ID":"d31d87b7-8bae-4f46-aab3-01dc25bf22e9","Type":"ContainerStarted","Data":"b2c2117146ff1684c8c4a90799ac2c903e7c345e48977a5fe8676b3bb646e956"} Jul 06 22:33:59.189449 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:33:59.189401 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" podStartSLOduration=2.012628822 podStartE2EDuration="2.189385802s" totalImagesPullingTime="176.75698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:33:59.187225676 +0000 UTC m=+571.491394364" watchObservedRunningTime="2026-07-06 22:33:59.189385802 +0000 UTC m=+571.493554472" Jul 06 22:34:08.317862 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:34:08.317829 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:34:08.318263 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:34:08.317892 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:34:08.330552 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:34:08.330531 2572 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:34:09.230780 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:34:09.230744 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dhhw7" Jul 06 22:34:28.102887 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:34:28.102860 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:34:28.110330 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:34:28.110308 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:36:42.802722 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:42.802690 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d57c96767-khxnr"] Jul 06 22:36:42.803270 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:42.802879 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-d57c96767-khxnr" podUID="5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" containerName="manager" containerID="cri-o://07dbadffa6f767a4fdae84d2b1fcf755cc673058f9d8611ce410982ab0a2e534" gracePeriod=30 Jul 06 22:36:42.965701 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:42.965670 2572 generic.go:356] "Generic (PLEG): container finished" podID="5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" containerID="07dbadffa6f767a4fdae84d2b1fcf755cc673058f9d8611ce410982ab0a2e534" exitCode=0 Jul 06 22:36:42.965837 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:42.965737 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d57c96767-khxnr" event={"ID":"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad","Type":"ContainerDied","Data":"07dbadffa6f767a4fdae84d2b1fcf755cc673058f9d8611ce410982ab0a2e534"} Jul 06 22:36:43.050826 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.050801 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:36:43.103776 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.103695 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqc5b\" (UniqueName: \"kubernetes.io/projected/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-kube-api-access-vqc5b\") pod \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " Jul 06 22:36:43.103905 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.103814 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-webhook-cert\") pod \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\" (UID: \"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad\") " Jul 06 22:36:43.106083 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.106053 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" (UID: "5059c1fe-c1f2-4ce2-a0cb-a6634858ccad"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:36:43.106083 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.106056 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-kube-api-access-vqc5b" (OuterVolumeSpecName: "kube-api-access-vqc5b") pod "5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" (UID: "5059c1fe-c1f2-4ce2-a0cb-a6634858ccad"). InnerVolumeSpecName "kube-api-access-vqc5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:36:43.204880 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.204847 2572 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-webhook-cert\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:36:43.204880 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.204876 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqc5b\" (UniqueName: \"kubernetes.io/projected/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad-kube-api-access-vqc5b\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:36:43.971690 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.971661 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d57c96767-khxnr" Jul 06 22:36:43.972143 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.971703 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d57c96767-khxnr" event={"ID":"5059c1fe-c1f2-4ce2-a0cb-a6634858ccad","Type":"ContainerDied","Data":"33179580e8d4537ce2a95e0e90ed50df9e2f569e4fee005ca1b62186bdc48cd7"} Jul 06 22:36:43.972143 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.971760 2572 scope.go:117] "RemoveContainer" containerID="07dbadffa6f767a4fdae84d2b1fcf755cc673058f9d8611ce410982ab0a2e534" Jul 06 22:36:43.994781 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.994755 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d57c96767-khxnr"] Jul 06 22:36:43.998187 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:43.998165 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d57c96767-khxnr"] Jul 06 22:36:44.248725 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.248641 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" path="/var/lib/kubelet/pods/5059c1fe-c1f2-4ce2-a0cb-a6634858ccad/volumes" Jul 06 22:36:44.749377 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.749344 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d57c96767-mhkrw"] Jul 06 22:36:44.749818 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.749802 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" containerName="manager" Jul 06 22:36:44.749859 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.749824 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" containerName="manager" Jul 06 22:36:44.749907 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.749898 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5059c1fe-c1f2-4ce2-a0cb-a6634858ccad" containerName="manager" Jul 06 22:36:44.754384 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.754363 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:44.758654 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.758632 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-f4vpz\"" Jul 06 22:36:44.758892 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.758641 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 22:36:44.759014 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.758641 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 22:36:44.761991 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.761967 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d57c96767-mhkrw"] Jul 06 22:36:44.819794 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.819757 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpdpb\" (UniqueName: \"kubernetes.io/projected/49cfd9ff-47de-4085-ab62-ec75ed383487-kube-api-access-bpdpb\") pod \"maas-controller-d57c96767-mhkrw\" (UID: \"49cfd9ff-47de-4085-ab62-ec75ed383487\") " pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:44.819978 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.819855 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49cfd9ff-47de-4085-ab62-ec75ed383487-webhook-cert\") pod \"maas-controller-d57c96767-mhkrw\" (UID: \"49cfd9ff-47de-4085-ab62-ec75ed383487\") " pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:44.920897 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.920861 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpdpb\" (UniqueName: \"kubernetes.io/projected/49cfd9ff-47de-4085-ab62-ec75ed383487-kube-api-access-bpdpb\") pod \"maas-controller-d57c96767-mhkrw\" (UID: \"49cfd9ff-47de-4085-ab62-ec75ed383487\") " pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:44.921066 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.920929 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49cfd9ff-47de-4085-ab62-ec75ed383487-webhook-cert\") pod \"maas-controller-d57c96767-mhkrw\" (UID: \"49cfd9ff-47de-4085-ab62-ec75ed383487\") " pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:44.923445 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.923425 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/49cfd9ff-47de-4085-ab62-ec75ed383487-webhook-cert\") pod \"maas-controller-d57c96767-mhkrw\" (UID: \"49cfd9ff-47de-4085-ab62-ec75ed383487\") " pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:44.928795 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:44.928773 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpdpb\" (UniqueName: \"kubernetes.io/projected/49cfd9ff-47de-4085-ab62-ec75ed383487-kube-api-access-bpdpb\") pod \"maas-controller-d57c96767-mhkrw\" (UID: \"49cfd9ff-47de-4085-ab62-ec75ed383487\") " pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:45.067536 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:45.067450 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:45.202008 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:45.201985 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d57c96767-mhkrw"] Jul 06 22:36:45.203703 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:36:45.203679 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49cfd9ff_47de_4085_ab62_ec75ed383487.slice/crio-9af1177877d7b1a22093ca275ff7be9c40902d964db83dee196bd66f8bb52016 WatchSource:0}: Error finding container 9af1177877d7b1a22093ca275ff7be9c40902d964db83dee196bd66f8bb52016: Status 404 returned error can't find the container with id 9af1177877d7b1a22093ca275ff7be9c40902d964db83dee196bd66f8bb52016 Jul 06 22:36:45.205057 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:45.205039 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:36:45.983927 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:45.983894 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d57c96767-mhkrw" event={"ID":"49cfd9ff-47de-4085-ab62-ec75ed383487","Type":"ContainerStarted","Data":"43cff004091101f7cb90dbc9b5036e3992f91f579a5c6da76eef3c44ebb2340d"} Jul 06 22:36:45.984113 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:45.983950 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d57c96767-mhkrw" event={"ID":"49cfd9ff-47de-4085-ab62-ec75ed383487","Type":"ContainerStarted","Data":"9af1177877d7b1a22093ca275ff7be9c40902d964db83dee196bd66f8bb52016"} Jul 06 22:36:45.984113 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:45.983979 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:36:46.002091 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:46.002037 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d57c96767-mhkrw" podStartSLOduration=1.432802849 podStartE2EDuration="2.002021446s" totalImagesPullingTime="569.218597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:36:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:36:45.999887687 +0000 UTC m=+738.304056341" watchObservedRunningTime="2026-07-06 22:36:46.002021446 +0000 UTC m=+738.306190104" Jul 06 22:36:56.994761 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:36:56.994724 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d57c96767-mhkrw" Jul 06 22:39:28.137356 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:39:28.137320 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:39:28.142271 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:39:28.142229 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:44:28.169779 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:44:28.169753 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:44:28.172170 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:44:28.172147 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:45:00.147366 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.147329 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-qzgzs"] Jul 06 22:45:00.150761 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.150744 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:00.167460 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.167426 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-qzgzs"] Jul 06 22:45:00.335106 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.335072 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgrvc\" (UniqueName: \"kubernetes.io/projected/217511b3-53ed-4fe0-a455-e931e28582c2-kube-api-access-lgrvc\") pod \"maas-api-key-cleanup-29722965-qzgzs\" (UID: \"217511b3-53ed-4fe0-a455-e931e28582c2\") " pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:00.436576 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.436543 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgrvc\" (UniqueName: \"kubernetes.io/projected/217511b3-53ed-4fe0-a455-e931e28582c2-kube-api-access-lgrvc\") pod \"maas-api-key-cleanup-29722965-qzgzs\" (UID: \"217511b3-53ed-4fe0-a455-e931e28582c2\") " pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:00.446951 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.446921 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgrvc\" (UniqueName: \"kubernetes.io/projected/217511b3-53ed-4fe0-a455-e931e28582c2-kube-api-access-lgrvc\") pod \"maas-api-key-cleanup-29722965-qzgzs\" (UID: \"217511b3-53ed-4fe0-a455-e931e28582c2\") " pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:00.460559 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.460538 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:00.590641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.590616 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722965-qzgzs"] Jul 06 22:45:00.593516 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:45:00.593489 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod217511b3_53ed_4fe0_a455_e931e28582c2.slice/crio-9430f2f1067a71c6f1bd039ef65bce5eff5dc1e935bcd1b5d467a8b1f0ed8045 WatchSource:0}: Error finding container 9430f2f1067a71c6f1bd039ef65bce5eff5dc1e935bcd1b5d467a8b1f0ed8045: Status 404 returned error can't find the container with id 9430f2f1067a71c6f1bd039ef65bce5eff5dc1e935bcd1b5d467a8b1f0ed8045 Jul 06 22:45:00.595693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:00.595677 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:45:01.463289 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:01.463257 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" event={"ID":"217511b3-53ed-4fe0-a455-e931e28582c2","Type":"ContainerStarted","Data":"9430f2f1067a71c6f1bd039ef65bce5eff5dc1e935bcd1b5d467a8b1f0ed8045"} Jul 06 22:45:03.477804 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:03.477761 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" event={"ID":"217511b3-53ed-4fe0-a455-e931e28582c2","Type":"ContainerStarted","Data":"77fb8221115f68c46b448f09f6acfe84ca66b3d8e88796b16a0fea907a330767"} Jul 06 22:45:03.495849 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:03.495798 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" podStartSLOduration=1.356847068 podStartE2EDuration="3.495783019s" totalImagesPullingTime="2.138935951s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:45:03.493422576 +0000 UTC m=+1235.797591237" watchObservedRunningTime="2026-07-06 22:45:03.495783019 +0000 UTC m=+1235.799951677" Jul 06 22:45:04.484140 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:04.484104 2572 generic.go:356] "Generic (PLEG): container finished" podID="217511b3-53ed-4fe0-a455-e931e28582c2" containerID="77fb8221115f68c46b448f09f6acfe84ca66b3d8e88796b16a0fea907a330767" exitCode=0 Jul 06 22:45:04.484604 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:04.484191 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" event={"ID":"217511b3-53ed-4fe0-a455-e931e28582c2","Type":"ContainerDied","Data":"77fb8221115f68c46b448f09f6acfe84ca66b3d8e88796b16a0fea907a330767"} Jul 06 22:45:05.621527 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:05.621498 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:05.691383 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:05.691348 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgrvc\" (UniqueName: \"kubernetes.io/projected/217511b3-53ed-4fe0-a455-e931e28582c2-kube-api-access-lgrvc\") pod \"217511b3-53ed-4fe0-a455-e931e28582c2\" (UID: \"217511b3-53ed-4fe0-a455-e931e28582c2\") " Jul 06 22:45:05.693492 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:05.693462 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/217511b3-53ed-4fe0-a455-e931e28582c2-kube-api-access-lgrvc" (OuterVolumeSpecName: "kube-api-access-lgrvc") pod "217511b3-53ed-4fe0-a455-e931e28582c2" (UID: "217511b3-53ed-4fe0-a455-e931e28582c2"). InnerVolumeSpecName "kube-api-access-lgrvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:45:05.792909 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:05.792818 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgrvc\" (UniqueName: \"kubernetes.io/projected/217511b3-53ed-4fe0-a455-e931e28582c2-kube-api-access-lgrvc\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:45:06.494135 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:06.494106 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" Jul 06 22:45:06.494315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:06.494153 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722965-qzgzs" event={"ID":"217511b3-53ed-4fe0-a455-e931e28582c2","Type":"ContainerDied","Data":"9430f2f1067a71c6f1bd039ef65bce5eff5dc1e935bcd1b5d467a8b1f0ed8045"} Jul 06 22:45:06.494315 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:45:06.494197 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9430f2f1067a71c6f1bd039ef65bce5eff5dc1e935bcd1b5d467a8b1f0ed8045" Jul 06 22:48:51.425474 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.425439 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls"] Jul 06 22:48:51.425997 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.425848 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="217511b3-53ed-4fe0-a455-e931e28582c2" containerName="cleanup" Jul 06 22:48:51.425997 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.425859 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="217511b3-53ed-4fe0-a455-e931e28582c2" containerName="cleanup" Jul 06 22:48:51.425997 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.425931 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="217511b3-53ed-4fe0-a455-e931e28582c2" containerName="cleanup" Jul 06 22:48:51.429140 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.429116 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.432156 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.432132 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-c265d48a-serving-cert\"" Jul 06 22:48:51.441167 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.441138 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls"] Jul 06 22:48:51.539007 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.538965 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8439b433-d1fd-4258-9ce4-f85fdf163dc3-openshift-service-ca\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.539154 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.539096 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zm28\" (UniqueName: \"kubernetes.io/projected/8439b433-d1fd-4258-9ce4-f85fdf163dc3-kube-api-access-2zm28\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.539214 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.539156 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.640688 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.640656 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8439b433-d1fd-4258-9ce4-f85fdf163dc3-openshift-service-ca\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.640874 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.640728 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zm28\" (UniqueName: \"kubernetes.io/projected/8439b433-d1fd-4258-9ce4-f85fdf163dc3-kube-api-access-2zm28\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.640874 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.640769 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.640990 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:48:51.640885 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-c265d48a-serving-cert: secret "maas-api-e2e-ait-c265d48a-serving-cert" not found Jul 06 22:48:51.640990 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:48:51.640942 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls podName:8439b433-d1fd-4258-9ce4-f85fdf163dc3 nodeName:}" failed. No retries permitted until 2026-07-06 22:48:52.140925788 +0000 UTC m=+1464.445094427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls") pod "maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" (UID: "8439b433-d1fd-4258-9ce4-f85fdf163dc3") : secret "maas-api-e2e-ait-c265d48a-serving-cert" not found Jul 06 22:48:51.641368 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.641349 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8439b433-d1fd-4258-9ce4-f85fdf163dc3-openshift-service-ca\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:51.650535 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:51.650506 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zm28\" (UniqueName: \"kubernetes.io/projected/8439b433-d1fd-4258-9ce4-f85fdf163dc3-kube-api-access-2zm28\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:52.144478 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:52.144439 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:52.146875 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:52.146855 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls\") pod \"maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:52.343405 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:52.343366 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:52.480984 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:52.480953 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls"] Jul 06 22:48:52.482491 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:48:52.482461 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8439b433_d1fd_4258_9ce4_f85fdf163dc3.slice/crio-6adc2ca020ba00e90f9b48baba4b2bc4b706fe4656854f47ed1b1fb5cedc274e WatchSource:0}: Error finding container 6adc2ca020ba00e90f9b48baba4b2bc4b706fe4656854f47ed1b1fb5cedc274e: Status 404 returned error can't find the container with id 6adc2ca020ba00e90f9b48baba4b2bc4b706fe4656854f47ed1b1fb5cedc274e Jul 06 22:48:52.618027 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:52.617937 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" event={"ID":"8439b433-d1fd-4258-9ce4-f85fdf163dc3","Type":"ContainerStarted","Data":"6adc2ca020ba00e90f9b48baba4b2bc4b706fe4656854f47ed1b1fb5cedc274e"} Jul 06 22:48:53.624719 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:53.624684 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" event={"ID":"8439b433-d1fd-4258-9ce4-f85fdf163dc3","Type":"ContainerStarted","Data":"3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8"} Jul 06 22:48:53.625084 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:53.624811 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:53.644104 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:53.644051 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" podStartSLOduration=2.117844322 podStartE2EDuration="2.644037164s" totalImagesPullingTime="526.192842ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:48:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:48:53.641888588 +0000 UTC m=+1465.946057263" watchObservedRunningTime="2026-07-06 22:48:53.644037164 +0000 UTC m=+1465.948205823" Jul 06 22:48:56.730180 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.730148 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls"] Jul 06 22:48:56.730770 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.730393 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" podUID="8439b433-d1fd-4258-9ce4-f85fdf163dc3" containerName="maas-api" containerID="cri-o://3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8" gracePeriod=30 Jul 06 22:48:56.980816 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.980757 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:56.983975 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.983957 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls\") pod \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " Jul 06 22:48:56.984087 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.984010 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8439b433-d1fd-4258-9ce4-f85fdf163dc3-openshift-service-ca\") pod \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " Jul 06 22:48:56.984151 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.984107 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zm28\" (UniqueName: \"kubernetes.io/projected/8439b433-d1fd-4258-9ce4-f85fdf163dc3-kube-api-access-2zm28\") pod \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\" (UID: \"8439b433-d1fd-4258-9ce4-f85fdf163dc3\") " Jul 06 22:48:56.984328 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.984305 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8439b433-d1fd-4258-9ce4-f85fdf163dc3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8439b433-d1fd-4258-9ce4-f85fdf163dc3" (UID: "8439b433-d1fd-4258-9ce4-f85fdf163dc3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:48:56.984463 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.984434 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8439b433-d1fd-4258-9ce4-f85fdf163dc3-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:48:56.986100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.986076 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8439b433-d1fd-4258-9ce4-f85fdf163dc3" (UID: "8439b433-d1fd-4258-9ce4-f85fdf163dc3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:48:56.986100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:56.986086 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8439b433-d1fd-4258-9ce4-f85fdf163dc3-kube-api-access-2zm28" (OuterVolumeSpecName: "kube-api-access-2zm28") pod "8439b433-d1fd-4258-9ce4-f85fdf163dc3" (UID: "8439b433-d1fd-4258-9ce4-f85fdf163dc3"). InnerVolumeSpecName "kube-api-access-2zm28". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:48:57.085708 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.085682 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8439b433-d1fd-4258-9ce4-f85fdf163dc3-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:48:57.085708 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.085706 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zm28\" (UniqueName: \"kubernetes.io/projected/8439b433-d1fd-4258-9ce4-f85fdf163dc3-kube-api-access-2zm28\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:48:57.645693 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.645659 2572 generic.go:356] "Generic (PLEG): container finished" podID="8439b433-d1fd-4258-9ce4-f85fdf163dc3" containerID="3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8" exitCode=0 Jul 06 22:48:57.645867 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.645701 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" Jul 06 22:48:57.645867 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.645719 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" event={"ID":"8439b433-d1fd-4258-9ce4-f85fdf163dc3","Type":"ContainerDied","Data":"3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8"} Jul 06 22:48:57.645867 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.645761 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls" event={"ID":"8439b433-d1fd-4258-9ce4-f85fdf163dc3","Type":"ContainerDied","Data":"6adc2ca020ba00e90f9b48baba4b2bc4b706fe4656854f47ed1b1fb5cedc274e"} Jul 06 22:48:57.645867 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.645783 2572 scope.go:117] "RemoveContainer" containerID="3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8" Jul 06 22:48:57.658190 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.658174 2572 scope.go:117] "RemoveContainer" containerID="3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8" Jul 06 22:48:57.658490 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:48:57.658469 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8\": container with ID starting with 3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8 not found: ID does not exist" containerID="3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8" Jul 06 22:48:57.658587 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.658499 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8"} err="failed to get container status \"3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8\": rpc error: code = NotFound desc = could not find container \"3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8\": container with ID starting with 3a9d36e0659be1932f49df14d2e1cfdc10cb8b6dd3181cfa8989650cea136ed8 not found: ID does not exist" Jul 06 22:48:57.671067 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.671042 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls"] Jul 06 22:48:57.672942 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:57.672923 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-c265d48a-5fbbcf44c8-q45ls"] Jul 06 22:48:58.249100 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:48:58.249065 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8439b433-d1fd-4258-9ce4-f85fdf163dc3" path="/var/lib/kubelet/pods/8439b433-d1fd-4258-9ce4-f85fdf163dc3/volumes" Jul 06 22:49:04.024466 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.024427 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg"] Jul 06 22:49:04.025073 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.025054 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8439b433-d1fd-4258-9ce4-f85fdf163dc3" containerName="maas-api" Jul 06 22:49:04.025115 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.025078 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8439b433-d1fd-4258-9ce4-f85fdf163dc3" containerName="maas-api" Jul 06 22:49:04.025216 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.025204 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="8439b433-d1fd-4258-9ce4-f85fdf163dc3" containerName="maas-api" Jul 06 22:49:04.030150 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.030128 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.033606 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.033584 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-e2eb8c66-serving-cert\"" Jul 06 22:49:04.036624 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.036596 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg"] Jul 06 22:49:04.051778 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.051747 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.051921 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.051856 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21bdb8bd-d3cd-4045-ae7e-99444f386458-openshift-service-ca\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.051969 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.051937 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7lr2\" (UniqueName: \"kubernetes.io/projected/21bdb8bd-d3cd-4045-ae7e-99444f386458-kube-api-access-c7lr2\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.152504 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.152470 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7lr2\" (UniqueName: \"kubernetes.io/projected/21bdb8bd-d3cd-4045-ae7e-99444f386458-kube-api-access-c7lr2\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.152705 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.152532 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.152705 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.152604 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21bdb8bd-d3cd-4045-ae7e-99444f386458-openshift-service-ca\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.152705 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:49:04.152691 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-e2eb8c66-serving-cert: secret "maas-api-e2e-ait-e2eb8c66-serving-cert" not found Jul 06 22:49:04.152878 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:49:04.152761 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls podName:21bdb8bd-d3cd-4045-ae7e-99444f386458 nodeName:}" failed. No retries permitted until 2026-07-06 22:49:04.652744005 +0000 UTC m=+1476.956912641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls") pod "maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" (UID: "21bdb8bd-d3cd-4045-ae7e-99444f386458") : secret "maas-api-e2e-ait-e2eb8c66-serving-cert" not found Jul 06 22:49:04.153274 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.153254 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21bdb8bd-d3cd-4045-ae7e-99444f386458-openshift-service-ca\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.161603 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.161571 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7lr2\" (UniqueName: \"kubernetes.io/projected/21bdb8bd-d3cd-4045-ae7e-99444f386458-kube-api-access-c7lr2\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.657900 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.657858 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.660520 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.660498 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls\") pod \"maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:04.942932 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:04.942895 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:05.082735 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:05.082706 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg"] Jul 06 22:49:05.085029 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:49:05.085001 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21bdb8bd_d3cd_4045_ae7e_99444f386458.slice/crio-0f0c0862dccd710731cbf77ffeeb894ef8c930e6608b9dc946ed25a2a6e5fec9 WatchSource:0}: Error finding container 0f0c0862dccd710731cbf77ffeeb894ef8c930e6608b9dc946ed25a2a6e5fec9: Status 404 returned error can't find the container with id 0f0c0862dccd710731cbf77ffeeb894ef8c930e6608b9dc946ed25a2a6e5fec9 Jul 06 22:49:05.692756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:05.692665 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" event={"ID":"21bdb8bd-d3cd-4045-ae7e-99444f386458","Type":"ContainerStarted","Data":"67e0fa3d74570558ff6bdb91ee64357cfaa83dbfed56145c6e2d2bf94688e537"} Jul 06 22:49:05.692756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:05.692727 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" event={"ID":"21bdb8bd-d3cd-4045-ae7e-99444f386458","Type":"ContainerStarted","Data":"0f0c0862dccd710731cbf77ffeeb894ef8c930e6608b9dc946ed25a2a6e5fec9"} Jul 06 22:49:05.692756 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:05.692756 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:05.709852 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:05.709803 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" podStartSLOduration=1.359156451 podStartE2EDuration="1.709787731s" totalImagesPullingTime="350.63128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:49:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:49:05.708767266 +0000 UTC m=+1478.012935937" watchObservedRunningTime="2026-07-06 22:49:05.709787731 +0000 UTC m=+1478.013956388" Jul 06 22:49:09.466418 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.466383 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg"] Jul 06 22:49:09.466778 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.466588 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" podUID="21bdb8bd-d3cd-4045-ae7e-99444f386458" containerName="maas-api" containerID="cri-o://67e0fa3d74570558ff6bdb91ee64357cfaa83dbfed56145c6e2d2bf94688e537" gracePeriod=30 Jul 06 22:49:09.714427 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.714389 2572 generic.go:356] "Generic (PLEG): container finished" podID="21bdb8bd-d3cd-4045-ae7e-99444f386458" containerID="67e0fa3d74570558ff6bdb91ee64357cfaa83dbfed56145c6e2d2bf94688e537" exitCode=0 Jul 06 22:49:09.714548 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.714454 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" event={"ID":"21bdb8bd-d3cd-4045-ae7e-99444f386458","Type":"ContainerDied","Data":"67e0fa3d74570558ff6bdb91ee64357cfaa83dbfed56145c6e2d2bf94688e537"} Jul 06 22:49:09.714548 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.714504 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" event={"ID":"21bdb8bd-d3cd-4045-ae7e-99444f386458","Type":"ContainerDied","Data":"0f0c0862dccd710731cbf77ffeeb894ef8c930e6608b9dc946ed25a2a6e5fec9"} Jul 06 22:49:09.714548 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.714526 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f0c0862dccd710731cbf77ffeeb894ef8c930e6608b9dc946ed25a2a6e5fec9" Jul 06 22:49:09.716209 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.716190 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:09.805592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.805504 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7lr2\" (UniqueName: \"kubernetes.io/projected/21bdb8bd-d3cd-4045-ae7e-99444f386458-kube-api-access-c7lr2\") pod \"21bdb8bd-d3cd-4045-ae7e-99444f386458\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " Jul 06 22:49:09.805592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.805561 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21bdb8bd-d3cd-4045-ae7e-99444f386458-openshift-service-ca\") pod \"21bdb8bd-d3cd-4045-ae7e-99444f386458\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " Jul 06 22:49:09.805592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.805589 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls\") pod \"21bdb8bd-d3cd-4045-ae7e-99444f386458\" (UID: \"21bdb8bd-d3cd-4045-ae7e-99444f386458\") " Jul 06 22:49:09.805940 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.805916 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21bdb8bd-d3cd-4045-ae7e-99444f386458-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21bdb8bd-d3cd-4045-ae7e-99444f386458" (UID: "21bdb8bd-d3cd-4045-ae7e-99444f386458"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:09.807728 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.807704 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21bdb8bd-d3cd-4045-ae7e-99444f386458" (UID: "21bdb8bd-d3cd-4045-ae7e-99444f386458"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:49:09.807816 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.807708 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21bdb8bd-d3cd-4045-ae7e-99444f386458-kube-api-access-c7lr2" (OuterVolumeSpecName: "kube-api-access-c7lr2") pod "21bdb8bd-d3cd-4045-ae7e-99444f386458" (UID: "21bdb8bd-d3cd-4045-ae7e-99444f386458"). InnerVolumeSpecName "kube-api-access-c7lr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:09.906999 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.906969 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7lr2\" (UniqueName: \"kubernetes.io/projected/21bdb8bd-d3cd-4045-ae7e-99444f386458-kube-api-access-c7lr2\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:49:09.906999 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.906998 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21bdb8bd-d3cd-4045-ae7e-99444f386458-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:49:09.906999 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:09.907007 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21bdb8bd-d3cd-4045-ae7e-99444f386458-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:49:10.719477 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:10.719445 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg" Jul 06 22:49:10.737298 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:10.737263 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg"] Jul 06 22:49:10.740303 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:10.740275 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-e2eb8c66-5d955d5b96-4gmhg"] Jul 06 22:49:12.247938 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:12.247905 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21bdb8bd-d3cd-4045-ae7e-99444f386458" path="/var/lib/kubelet/pods/21bdb8bd-d3cd-4045-ae7e-99444f386458/volumes" Jul 06 22:49:22.803145 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.803116 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf"] Jul 06 22:49:22.803561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.803537 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21bdb8bd-d3cd-4045-ae7e-99444f386458" containerName="maas-api" Jul 06 22:49:22.803561 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.803550 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="21bdb8bd-d3cd-4045-ae7e-99444f386458" containerName="maas-api" Jul 06 22:49:22.803679 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.803667 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="21bdb8bd-d3cd-4045-ae7e-99444f386458" containerName="maas-api" Jul 06 22:49:22.807058 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.807040 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:22.810983 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.810964 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-08e547ba-serving-cert\"" Jul 06 22:49:22.816685 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.816662 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf"] Jul 06 22:49:22.936419 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.936387 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af15f6bc-31e9-4a85-918e-0aafef28ca5c-openshift-service-ca\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:22.936580 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.936459 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:22.936580 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:22.936499 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpwvk\" (UniqueName: \"kubernetes.io/projected/af15f6bc-31e9-4a85-918e-0aafef28ca5c-kube-api-access-hpwvk\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.037997 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.037960 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpwvk\" (UniqueName: \"kubernetes.io/projected/af15f6bc-31e9-4a85-918e-0aafef28ca5c-kube-api-access-hpwvk\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.038186 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.038026 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af15f6bc-31e9-4a85-918e-0aafef28ca5c-openshift-service-ca\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.038186 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.038074 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.038186 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:49:23.038163 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-08e547ba-serving-cert: secret "maas-api-e2e-derive-08e547ba-serving-cert" not found Jul 06 22:49:23.038377 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:49:23.038227 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls podName:af15f6bc-31e9-4a85-918e-0aafef28ca5c nodeName:}" failed. No retries permitted until 2026-07-06 22:49:23.538211875 +0000 UTC m=+1495.842380516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls") pod "maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" (UID: "af15f6bc-31e9-4a85-918e-0aafef28ca5c") : secret "maas-api-e2e-derive-08e547ba-serving-cert" not found Jul 06 22:49:23.038661 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.038643 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af15f6bc-31e9-4a85-918e-0aafef28ca5c-openshift-service-ca\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.047256 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.047210 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpwvk\" (UniqueName: \"kubernetes.io/projected/af15f6bc-31e9-4a85-918e-0aafef28ca5c-kube-api-access-hpwvk\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.543514 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.543480 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.545956 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.545937 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls\") pod \"maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.719453 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.719418 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:23.861384 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:23.861360 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf"] Jul 06 22:49:23.863462 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:49:23.863437 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf15f6bc_31e9_4a85_918e_0aafef28ca5c.slice/crio-a1b147f85b1cfa3b8949830ab1ed7ff7399f6034cd7ee7d1f91ffda37b077af9 WatchSource:0}: Error finding container a1b147f85b1cfa3b8949830ab1ed7ff7399f6034cd7ee7d1f91ffda37b077af9: Status 404 returned error can't find the container with id a1b147f85b1cfa3b8949830ab1ed7ff7399f6034cd7ee7d1f91ffda37b077af9 Jul 06 22:49:24.787921 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:24.787883 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" event={"ID":"af15f6bc-31e9-4a85-918e-0aafef28ca5c","Type":"ContainerStarted","Data":"dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02"} Jul 06 22:49:24.788086 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:24.787936 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" event={"ID":"af15f6bc-31e9-4a85-918e-0aafef28ca5c","Type":"ContainerStarted","Data":"a1b147f85b1cfa3b8949830ab1ed7ff7399f6034cd7ee7d1f91ffda37b077af9"} Jul 06 22:49:24.788086 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:24.787966 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:24.804732 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:24.804636 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" podStartSLOduration=2.542533141 podStartE2EDuration="2.804616668s" totalImagesPullingTime="262.083527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:49:24.803788321 +0000 UTC m=+1497.107956980" watchObservedRunningTime="2026-07-06 22:49:24.804616668 +0000 UTC m=+1497.108785349" Jul 06 22:49:27.406493 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.406462 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf"] Jul 06 22:49:27.406944 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.406643 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" podUID="af15f6bc-31e9-4a85-918e-0aafef28ca5c" containerName="maas-api" containerID="cri-o://dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02" gracePeriod=30 Jul 06 22:49:27.662992 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.662934 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:27.783598 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.783571 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hpwvk\" (UniqueName: \"kubernetes.io/projected/af15f6bc-31e9-4a85-918e-0aafef28ca5c-kube-api-access-hpwvk\") pod \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " Jul 06 22:49:27.783598 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.783603 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af15f6bc-31e9-4a85-918e-0aafef28ca5c-openshift-service-ca\") pod \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " Jul 06 22:49:27.783791 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.783629 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls\") pod \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\" (UID: \"af15f6bc-31e9-4a85-918e-0aafef28ca5c\") " Jul 06 22:49:27.784042 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.784020 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af15f6bc-31e9-4a85-918e-0aafef28ca5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af15f6bc-31e9-4a85-918e-0aafef28ca5c" (UID: "af15f6bc-31e9-4a85-918e-0aafef28ca5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:27.785858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.785833 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af15f6bc-31e9-4a85-918e-0aafef28ca5c-kube-api-access-hpwvk" (OuterVolumeSpecName: "kube-api-access-hpwvk") pod "af15f6bc-31e9-4a85-918e-0aafef28ca5c" (UID: "af15f6bc-31e9-4a85-918e-0aafef28ca5c"). InnerVolumeSpecName "kube-api-access-hpwvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:27.785858 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.785843 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af15f6bc-31e9-4a85-918e-0aafef28ca5c" (UID: "af15f6bc-31e9-4a85-918e-0aafef28ca5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:49:27.803483 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.803458 2572 generic.go:356] "Generic (PLEG): container finished" podID="af15f6bc-31e9-4a85-918e-0aafef28ca5c" containerID="dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02" exitCode=0 Jul 06 22:49:27.803584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.803496 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" event={"ID":"af15f6bc-31e9-4a85-918e-0aafef28ca5c","Type":"ContainerDied","Data":"dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02"} Jul 06 22:49:27.803584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.803504 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" Jul 06 22:49:27.803584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.803560 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf" event={"ID":"af15f6bc-31e9-4a85-918e-0aafef28ca5c","Type":"ContainerDied","Data":"a1b147f85b1cfa3b8949830ab1ed7ff7399f6034cd7ee7d1f91ffda37b077af9"} Jul 06 22:49:27.803584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.803580 2572 scope.go:117] "RemoveContainer" containerID="dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02" Jul 06 22:49:27.813301 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.813280 2572 scope.go:117] "RemoveContainer" containerID="dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02" Jul 06 22:49:27.813547 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:49:27.813528 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02\": container with ID starting with dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02 not found: ID does not exist" containerID="dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02" Jul 06 22:49:27.813610 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.813563 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02"} err="failed to get container status \"dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02\": rpc error: code = NotFound desc = could not find container \"dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02\": container with ID starting with dcfb242c1ba3dfbbf4743b79447bf5a2eb49eba76a2b0f90cf94207c33cadf02 not found: ID does not exist" Jul 06 22:49:27.824511 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.824486 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf"] Jul 06 22:49:27.827948 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.827928 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-08e547ba-bbb8d7f76-kh4bf"] Jul 06 22:49:27.885329 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.885305 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hpwvk\" (UniqueName: \"kubernetes.io/projected/af15f6bc-31e9-4a85-918e-0aafef28ca5c-kube-api-access-hpwvk\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:49:27.885329 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.885329 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af15f6bc-31e9-4a85-918e-0aafef28ca5c-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:49:27.885329 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:27.885339 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af15f6bc-31e9-4a85-918e-0aafef28ca5c-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:49:28.204171 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:28.204146 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:49:28.204459 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:28.204434 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:49:28.248564 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:49:28.248533 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af15f6bc-31e9-4a85-918e-0aafef28ca5c" path="/var/lib/kubelet/pods/af15f6bc-31e9-4a85-918e-0aafef28ca5c/volumes" Jul 06 22:51:11.943478 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.943436 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24"] Jul 06 22:51:11.943960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.943913 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af15f6bc-31e9-4a85-918e-0aafef28ca5c" containerName="maas-api" Jul 06 22:51:11.943960 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.943927 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="af15f6bc-31e9-4a85-918e-0aafef28ca5c" containerName="maas-api" Jul 06 22:51:11.944049 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.943995 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="af15f6bc-31e9-4a85-918e-0aafef28ca5c" containerName="maas-api" Jul 06 22:51:11.947392 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.947353 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:11.950165 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.950137 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-50bd30f5-serving-cert\"" Jul 06 22:51:11.958909 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:11.958881 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24"] Jul 06 22:51:12.073569 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.073527 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.073569 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.073570 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04139435-2601-47a2-ab21-81d1f769282e-openshift-service-ca\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.073775 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.073686 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfrw7\" (UniqueName: \"kubernetes.io/projected/04139435-2601-47a2-ab21-81d1f769282e-kube-api-access-mfrw7\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.174741 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.174703 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfrw7\" (UniqueName: \"kubernetes.io/projected/04139435-2601-47a2-ab21-81d1f769282e-kube-api-access-mfrw7\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.174917 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.174819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.174917 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.174848 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04139435-2601-47a2-ab21-81d1f769282e-openshift-service-ca\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.175015 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:51:12.174965 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-50bd30f5-serving-cert: secret "maas-api-e2e-mt-50bd30f5-serving-cert" not found Jul 06 22:51:12.175056 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:51:12.175038 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls podName:04139435-2601-47a2-ab21-81d1f769282e nodeName:}" failed. No retries permitted until 2026-07-06 22:51:12.675021957 +0000 UTC m=+1604.979190596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls") pod "maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" (UID: "04139435-2601-47a2-ab21-81d1f769282e") : secret "maas-api-e2e-mt-50bd30f5-serving-cert" not found Jul 06 22:51:12.175539 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.175521 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04139435-2601-47a2-ab21-81d1f769282e-openshift-service-ca\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.185582 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.185553 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfrw7\" (UniqueName: \"kubernetes.io/projected/04139435-2601-47a2-ab21-81d1f769282e-kube-api-access-mfrw7\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.680427 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.680384 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.683073 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.683045 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls\") pod \"maas-api-e2e-mt-50bd30f5-77869ffb7-69d24\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:12.860254 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:12.860193 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:13.005515 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:13.005476 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24"] Jul 06 22:51:13.009718 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:51:13.009681 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04139435_2601_47a2_ab21_81d1f769282e.slice/crio-77f0d36fafec6b1b6c4f0fb3958dc9747a036e8d403c46c9308a42d3de08ea1f WatchSource:0}: Error finding container 77f0d36fafec6b1b6c4f0fb3958dc9747a036e8d403c46c9308a42d3de08ea1f: Status 404 returned error can't find the container with id 77f0d36fafec6b1b6c4f0fb3958dc9747a036e8d403c46c9308a42d3de08ea1f Jul 06 22:51:13.011553 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:13.011530 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:51:13.333681 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:13.333583 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" event={"ID":"04139435-2601-47a2-ab21-81d1f769282e","Type":"ContainerStarted","Data":"77f0d36fafec6b1b6c4f0fb3958dc9747a036e8d403c46c9308a42d3de08ea1f"} Jul 06 22:51:14.340606 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:14.340560 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" event={"ID":"04139435-2601-47a2-ab21-81d1f769282e","Type":"ContainerStarted","Data":"9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f"} Jul 06 22:51:14.340989 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:14.340717 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:14.358895 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:14.358845 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" podStartSLOduration=2.940973644 podStartE2EDuration="3.358829047s" totalImagesPullingTime="417.855403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:51:14.356375535 +0000 UTC m=+1606.660544199" watchObservedRunningTime="2026-07-06 22:51:14.358829047 +0000 UTC m=+1606.662997728" Jul 06 22:51:20.351640 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:20.351613 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:24.108141 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.108109 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24"] Jul 06 22:51:24.108580 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.108308 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" podUID="04139435-2601-47a2-ab21-81d1f769282e" containerName="maas-api" containerID="cri-o://9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f" gracePeriod=30 Jul 06 22:51:24.360477 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.360420 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:24.390153 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.390117 2572 generic.go:356] "Generic (PLEG): container finished" podID="04139435-2601-47a2-ab21-81d1f769282e" containerID="9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f" exitCode=0 Jul 06 22:51:24.390314 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.390161 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" Jul 06 22:51:24.390314 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.390203 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" event={"ID":"04139435-2601-47a2-ab21-81d1f769282e","Type":"ContainerDied","Data":"9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f"} Jul 06 22:51:24.390314 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.390280 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24" event={"ID":"04139435-2601-47a2-ab21-81d1f769282e","Type":"ContainerDied","Data":"77f0d36fafec6b1b6c4f0fb3958dc9747a036e8d403c46c9308a42d3de08ea1f"} Jul 06 22:51:24.390753 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.390314 2572 scope.go:117] "RemoveContainer" containerID="9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f" Jul 06 22:51:24.400867 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.400845 2572 scope.go:117] "RemoveContainer" containerID="9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f" Jul 06 22:51:24.401130 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:51:24.401113 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f\": container with ID starting with 9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f not found: ID does not exist" containerID="9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f" Jul 06 22:51:24.401194 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.401143 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f"} err="failed to get container status \"9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f\": rpc error: code = NotFound desc = could not find container \"9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f\": container with ID starting with 9eaaa57982b7cbc4034983484a63306bbe71da48b045d785597e87afbdeae68f not found: ID does not exist" Jul 06 22:51:24.493903 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.493867 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls\") pod \"04139435-2601-47a2-ab21-81d1f769282e\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " Jul 06 22:51:24.494066 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.494002 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfrw7\" (UniqueName: \"kubernetes.io/projected/04139435-2601-47a2-ab21-81d1f769282e-kube-api-access-mfrw7\") pod \"04139435-2601-47a2-ab21-81d1f769282e\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " Jul 06 22:51:24.494066 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.494032 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04139435-2601-47a2-ab21-81d1f769282e-openshift-service-ca\") pod \"04139435-2601-47a2-ab21-81d1f769282e\" (UID: \"04139435-2601-47a2-ab21-81d1f769282e\") " Jul 06 22:51:24.494468 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.494435 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04139435-2601-47a2-ab21-81d1f769282e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04139435-2601-47a2-ab21-81d1f769282e" (UID: "04139435-2601-47a2-ab21-81d1f769282e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:51:24.496179 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.496158 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04139435-2601-47a2-ab21-81d1f769282e" (UID: "04139435-2601-47a2-ab21-81d1f769282e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:51:24.496284 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.496196 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04139435-2601-47a2-ab21-81d1f769282e-kube-api-access-mfrw7" (OuterVolumeSpecName: "kube-api-access-mfrw7") pod "04139435-2601-47a2-ab21-81d1f769282e" (UID: "04139435-2601-47a2-ab21-81d1f769282e"). InnerVolumeSpecName "kube-api-access-mfrw7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:51:24.595642 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.595603 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04139435-2601-47a2-ab21-81d1f769282e-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:51:24.595642 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.595635 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfrw7\" (UniqueName: \"kubernetes.io/projected/04139435-2601-47a2-ab21-81d1f769282e-kube-api-access-mfrw7\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:51:24.595642 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.595644 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04139435-2601-47a2-ab21-81d1f769282e-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:51:24.716860 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.716826 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24"] Jul 06 22:51:24.721025 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:24.721000 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-50bd30f5-77869ffb7-69d24"] Jul 06 22:51:26.248680 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:26.248648 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04139435-2601-47a2-ab21-81d1f769282e" path="/var/lib/kubelet/pods/04139435-2601-47a2-ab21-81d1f769282e/volumes" Jul 06 22:51:52.958768 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.958732 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b"] Jul 06 22:51:52.959394 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.959142 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04139435-2601-47a2-ab21-81d1f769282e" containerName="maas-api" Jul 06 22:51:52.959394 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.959153 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="04139435-2601-47a2-ab21-81d1f769282e" containerName="maas-api" Jul 06 22:51:52.959394 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.959204 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="04139435-2601-47a2-ab21-81d1f769282e" containerName="maas-api" Jul 06 22:51:52.963626 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.963606 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:52.966375 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.966354 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-fed91589-serving-cert\"" Jul 06 22:51:52.972769 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:52.972748 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b"] Jul 06 22:51:53.063352 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.063311 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxcx8\" (UniqueName: \"kubernetes.io/projected/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-kube-api-access-xxcx8\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.063352 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.063357 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.063574 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.063381 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-openshift-service-ca\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.164112 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.164075 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxcx8\" (UniqueName: \"kubernetes.io/projected/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-kube-api-access-xxcx8\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.164112 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.164114 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.164366 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.164139 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-openshift-service-ca\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.164366 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:51:53.164270 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-fed91589-serving-cert: secret "maas-api-e2e-mt-fed91589-serving-cert" not found Jul 06 22:51:53.164366 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:51:53.164333 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls podName:35d7b003-4ac2-439d-ba5b-11dc789bfe2d nodeName:}" failed. No retries permitted until 2026-07-06 22:51:53.664316867 +0000 UTC m=+1645.968485508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls") pod "maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" (UID: "35d7b003-4ac2-439d-ba5b-11dc789bfe2d") : secret "maas-api-e2e-mt-fed91589-serving-cert" not found Jul 06 22:51:53.164792 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.164775 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-openshift-service-ca\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.176034 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.176018 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxcx8\" (UniqueName: \"kubernetes.io/projected/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-kube-api-access-xxcx8\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.668862 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.668819 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.671535 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.671512 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls\") pod \"maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:53.875155 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:53.875113 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:54.025812 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:54.025788 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b"] Jul 06 22:51:54.027921 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:51:54.027890 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d7b003_4ac2_439d_ba5b_11dc789bfe2d.slice/crio-bf3c838752606d4a9522832567ec64b1fe0a80f0a20376cca865a5f5fe985c40 WatchSource:0}: Error finding container bf3c838752606d4a9522832567ec64b1fe0a80f0a20376cca865a5f5fe985c40: Status 404 returned error can't find the container with id bf3c838752606d4a9522832567ec64b1fe0a80f0a20376cca865a5f5fe985c40 Jul 06 22:51:54.557989 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:54.557953 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" event={"ID":"35d7b003-4ac2-439d-ba5b-11dc789bfe2d","Type":"ContainerStarted","Data":"b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044"} Jul 06 22:51:54.558157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:54.558011 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" event={"ID":"35d7b003-4ac2-439d-ba5b-11dc789bfe2d","Type":"ContainerStarted","Data":"bf3c838752606d4a9522832567ec64b1fe0a80f0a20376cca865a5f5fe985c40"} Jul 06 22:51:54.558157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:54.558068 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:51:54.578997 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:51:54.578949 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" podStartSLOduration=2.18984316 podStartE2EDuration="2.57893321s" totalImagesPullingTime="389.09005ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:51:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:51:54.576368266 +0000 UTC m=+1646.880536925" watchObservedRunningTime="2026-07-06 22:51:54.57893321 +0000 UTC m=+1646.883101869" Jul 06 22:52:00.567691 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:00.567665 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:52:30.101848 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.101815 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b"] Jul 06 22:52:30.102353 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.102014 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" podUID="35d7b003-4ac2-439d-ba5b-11dc789bfe2d" containerName="maas-api" containerID="cri-o://b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044" gracePeriod=30 Jul 06 22:52:30.355890 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.355829 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:52:30.412578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.412551 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-openshift-service-ca\") pod \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " Jul 06 22:52:30.412760 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.412586 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxcx8\" (UniqueName: \"kubernetes.io/projected/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-kube-api-access-xxcx8\") pod \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " Jul 06 22:52:30.412760 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.412727 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls\") pod \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\" (UID: \"35d7b003-4ac2-439d-ba5b-11dc789bfe2d\") " Jul 06 22:52:30.412974 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.412950 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35d7b003-4ac2-439d-ba5b-11dc789bfe2d" (UID: "35d7b003-4ac2-439d-ba5b-11dc789bfe2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:52:30.413039 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.413016 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:52:30.414823 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.414800 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35d7b003-4ac2-439d-ba5b-11dc789bfe2d" (UID: "35d7b003-4ac2-439d-ba5b-11dc789bfe2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:52:30.414899 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.414866 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-kube-api-access-xxcx8" (OuterVolumeSpecName: "kube-api-access-xxcx8") pod "35d7b003-4ac2-439d-ba5b-11dc789bfe2d" (UID: "35d7b003-4ac2-439d-ba5b-11dc789bfe2d"). InnerVolumeSpecName "kube-api-access-xxcx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:52:30.514480 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.514451 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxcx8\" (UniqueName: \"kubernetes.io/projected/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-kube-api-access-xxcx8\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:52:30.514480 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.514477 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d7b003-4ac2-439d-ba5b-11dc789bfe2d-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:52:30.743636 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.743607 2572 generic.go:356] "Generic (PLEG): container finished" podID="35d7b003-4ac2-439d-ba5b-11dc789bfe2d" containerID="b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044" exitCode=0 Jul 06 22:52:30.743806 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.743662 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" Jul 06 22:52:30.743806 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.743683 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" event={"ID":"35d7b003-4ac2-439d-ba5b-11dc789bfe2d","Type":"ContainerDied","Data":"b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044"} Jul 06 22:52:30.743806 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.743726 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b" event={"ID":"35d7b003-4ac2-439d-ba5b-11dc789bfe2d","Type":"ContainerDied","Data":"bf3c838752606d4a9522832567ec64b1fe0a80f0a20376cca865a5f5fe985c40"} Jul 06 22:52:30.743806 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.743748 2572 scope.go:117] "RemoveContainer" containerID="b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044" Jul 06 22:52:30.754013 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.753992 2572 scope.go:117] "RemoveContainer" containerID="b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044" Jul 06 22:52:30.754467 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:52:30.754331 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044\": container with ID starting with b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044 not found: ID does not exist" containerID="b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044" Jul 06 22:52:30.754467 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.754361 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044"} err="failed to get container status \"b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044\": rpc error: code = NotFound desc = could not find container \"b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044\": container with ID starting with b1bf71fd3a0fd1d48036abf3229b07365307031a8dfecc370ba2f89c69b3f044 not found: ID does not exist" Jul 06 22:52:30.768805 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.768774 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b"] Jul 06 22:52:30.772469 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:30.772439 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-fed91589-5bf9ddbd8f-4pk5b"] Jul 06 22:52:32.248187 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:52:32.248145 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35d7b003-4ac2-439d-ba5b-11dc789bfe2d" path="/var/lib/kubelet/pods/35d7b003-4ac2-439d-ba5b-11dc789bfe2d/volumes" Jul 06 22:54:21.870170 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.870140 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff"] Jul 06 22:54:21.870777 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.870757 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35d7b003-4ac2-439d-ba5b-11dc789bfe2d" containerName="maas-api" Jul 06 22:54:21.870862 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.870781 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="35d7b003-4ac2-439d-ba5b-11dc789bfe2d" containerName="maas-api" Jul 06 22:54:21.870916 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.870892 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="35d7b003-4ac2-439d-ba5b-11dc789bfe2d" containerName="maas-api" Jul 06 22:54:21.873963 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.873944 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:21.879001 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.878983 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-b673d1-serving-cert\"" Jul 06 22:54:21.905666 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.905638 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff"] Jul 06 22:54:21.976965 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.976937 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40599807-8c82-415f-a07e-fa3ad0d24df1-maas-api-tls\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:21.977132 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.977003 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvw8d\" (UniqueName: \"kubernetes.io/projected/40599807-8c82-415f-a07e-fa3ad0d24df1-kube-api-access-mvw8d\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:21.977132 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:21.977069 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40599807-8c82-415f-a07e-fa3ad0d24df1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.078133 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.078103 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvw8d\" (UniqueName: \"kubernetes.io/projected/40599807-8c82-415f-a07e-fa3ad0d24df1-kube-api-access-mvw8d\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.078339 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.078178 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40599807-8c82-415f-a07e-fa3ad0d24df1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.078339 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.078305 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40599807-8c82-415f-a07e-fa3ad0d24df1-maas-api-tls\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.078827 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.078803 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40599807-8c82-415f-a07e-fa3ad0d24df1-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.080670 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.080651 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40599807-8c82-415f-a07e-fa3ad0d24df1-maas-api-tls\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.086964 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.086935 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvw8d\" (UniqueName: \"kubernetes.io/projected/40599807-8c82-415f-a07e-fa3ad0d24df1-kube-api-access-mvw8d\") pod \"maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.185239 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.185212 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:22.323776 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:22.323652 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff"] Jul 06 22:54:22.325984 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:54:22.325956 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40599807_8c82_415f_a07e_fa3ad0d24df1.slice/crio-d9494e446db53812986c1e13f30a9717cbab6ca7de82304d31d4ee7fb0d191c6 WatchSource:0}: Error finding container d9494e446db53812986c1e13f30a9717cbab6ca7de82304d31d4ee7fb0d191c6: Status 404 returned error can't find the container with id d9494e446db53812986c1e13f30a9717cbab6ca7de82304d31d4ee7fb0d191c6 Jul 06 22:54:23.329432 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:23.329390 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" event={"ID":"40599807-8c82-415f-a07e-fa3ad0d24df1","Type":"ContainerStarted","Data":"da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907"} Jul 06 22:54:23.329810 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:23.329445 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" event={"ID":"40599807-8c82-415f-a07e-fa3ad0d24df1","Type":"ContainerStarted","Data":"d9494e446db53812986c1e13f30a9717cbab6ca7de82304d31d4ee7fb0d191c6"} Jul 06 22:54:23.329810 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:23.329504 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:23.347780 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:23.347736 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" podStartSLOduration=1.8613139109999999 podStartE2EDuration="2.347721314s" totalImagesPullingTime="486.407403ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:23.34553183 +0000 UTC m=+1795.649700489" watchObservedRunningTime="2026-07-06 22:54:23.347721314 +0000 UTC m=+1795.651889976" Jul 06 22:54:28.238136 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:28.238109 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:54:28.239403 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:28.239385 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:54:29.339546 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.339519 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:54:29.744051 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.744014 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx"] Jul 06 22:54:29.748126 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.748105 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.751016 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.750991 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-f49549-serving-cert\"" Jul 06 22:54:29.753278 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.753253 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gkvj\" (UniqueName: \"kubernetes.io/projected/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-kube-api-access-4gkvj\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.753498 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.753477 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.754020 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.753998 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.765661 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.765615 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx"] Jul 06 22:54:29.855425 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.855396 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gkvj\" (UniqueName: \"kubernetes.io/projected/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-kube-api-access-4gkvj\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.855592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.855448 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.855592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.855514 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.855708 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:54:29.855599 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-f49549-serving-cert: secret "maas-api-e2e-inf-b-f49549-serving-cert" not found Jul 06 22:54:29.855708 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:54:29.855652 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls podName:5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f nodeName:}" failed. No retries permitted until 2026-07-06 22:54:30.355636651 +0000 UTC m=+1802.659805290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls") pod "maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" (UID: "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f") : secret "maas-api-e2e-inf-b-f49549-serving-cert" not found Jul 06 22:54:29.856144 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.856120 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:29.865715 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:29.865687 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gkvj\" (UniqueName: \"kubernetes.io/projected/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-kube-api-access-4gkvj\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:30.360120 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:30.360088 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:30.363172 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:30.363146 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls\") pod \"maas-api-e2e-inf-b-f49549-987c84cbd-5jssx\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:30.371059 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:30.371038 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:30.516668 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:30.516642 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx"] Jul 06 22:54:30.518968 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:54:30.518939 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dbe1ca0_e18f_428f_b66d_c4753e6fbb9f.slice/crio-97cd0c085689d36dd2de9fe5512c4f40ac6cecba0fb6fdafd3d6dd8c4c6e3a24 WatchSource:0}: Error finding container 97cd0c085689d36dd2de9fe5512c4f40ac6cecba0fb6fdafd3d6dd8c4c6e3a24: Status 404 returned error can't find the container with id 97cd0c085689d36dd2de9fe5512c4f40ac6cecba0fb6fdafd3d6dd8c4c6e3a24 Jul 06 22:54:31.375346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:31.375267 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" event={"ID":"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f","Type":"ContainerStarted","Data":"6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733"} Jul 06 22:54:31.375346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:31.375318 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" event={"ID":"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f","Type":"ContainerStarted","Data":"97cd0c085689d36dd2de9fe5512c4f40ac6cecba0fb6fdafd3d6dd8c4c6e3a24"} Jul 06 22:54:31.375710 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:31.375377 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:54:31.392408 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:31.392358 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" podStartSLOduration=1.792652816 podStartE2EDuration="2.392343681s" totalImagesPullingTime="599.690865ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:54:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:54:31.391571682 +0000 UTC m=+1803.695740343" watchObservedRunningTime="2026-07-06 22:54:31.392343681 +0000 UTC m=+1803.696512338" Jul 06 22:54:37.386476 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:54:37.386438 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:55:28.313665 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:28.313629 2572 scope.go:117] "RemoveContainer" containerID="67e0fa3d74570558ff6bdb91ee64357cfaa83dbfed56145c6e2d2bf94688e537" Jul 06 22:55:29.152558 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.152521 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff"] Jul 06 22:55:29.152823 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.152766 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" podUID="40599807-8c82-415f-a07e-fa3ad0d24df1" containerName="maas-api" containerID="cri-o://da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907" gracePeriod=30 Jul 06 22:55:29.335904 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.335867 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" podUID="40599807-8c82-415f-a07e-fa3ad0d24df1" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.46:8443/health\": dial tcp 10.134.0.46:8443: connect: connection refused" Jul 06 22:55:29.606633 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.606607 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:55:29.675534 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.675501 2572 generic.go:356] "Generic (PLEG): container finished" podID="40599807-8c82-415f-a07e-fa3ad0d24df1" containerID="da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907" exitCode=0 Jul 06 22:55:29.676074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.675549 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" Jul 06 22:55:29.676074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.675551 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" event={"ID":"40599807-8c82-415f-a07e-fa3ad0d24df1","Type":"ContainerDied","Data":"da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907"} Jul 06 22:55:29.676074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.675682 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff" event={"ID":"40599807-8c82-415f-a07e-fa3ad0d24df1","Type":"ContainerDied","Data":"d9494e446db53812986c1e13f30a9717cbab6ca7de82304d31d4ee7fb0d191c6"} Jul 06 22:55:29.676074 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.675710 2572 scope.go:117] "RemoveContainer" containerID="da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907" Jul 06 22:55:29.684341 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.684325 2572 scope.go:117] "RemoveContainer" containerID="da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907" Jul 06 22:55:29.684600 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:55:29.684584 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907\": container with ID starting with da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907 not found: ID does not exist" containerID="da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907" Jul 06 22:55:29.684648 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.684608 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907"} err="failed to get container status \"da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907\": rpc error: code = NotFound desc = could not find container \"da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907\": container with ID starting with da369025f64965a158fcf573ce7e028228e8f24fc027b55fbaab222eb3499907 not found: ID does not exist" Jul 06 22:55:29.750047 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.749961 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40599807-8c82-415f-a07e-fa3ad0d24df1-openshift-service-ca\") pod \"40599807-8c82-415f-a07e-fa3ad0d24df1\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " Jul 06 22:55:29.750047 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.750001 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40599807-8c82-415f-a07e-fa3ad0d24df1-maas-api-tls\") pod \"40599807-8c82-415f-a07e-fa3ad0d24df1\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " Jul 06 22:55:29.750283 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.750071 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mvw8d\" (UniqueName: \"kubernetes.io/projected/40599807-8c82-415f-a07e-fa3ad0d24df1-kube-api-access-mvw8d\") pod \"40599807-8c82-415f-a07e-fa3ad0d24df1\" (UID: \"40599807-8c82-415f-a07e-fa3ad0d24df1\") " Jul 06 22:55:29.750396 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.750373 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40599807-8c82-415f-a07e-fa3ad0d24df1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "40599807-8c82-415f-a07e-fa3ad0d24df1" (UID: "40599807-8c82-415f-a07e-fa3ad0d24df1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:55:29.750466 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.750438 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/40599807-8c82-415f-a07e-fa3ad0d24df1-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:55:29.752267 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.752215 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/40599807-8c82-415f-a07e-fa3ad0d24df1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "40599807-8c82-415f-a07e-fa3ad0d24df1" (UID: "40599807-8c82-415f-a07e-fa3ad0d24df1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:55:29.752385 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.752350 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40599807-8c82-415f-a07e-fa3ad0d24df1-kube-api-access-mvw8d" (OuterVolumeSpecName: "kube-api-access-mvw8d") pod "40599807-8c82-415f-a07e-fa3ad0d24df1" (UID: "40599807-8c82-415f-a07e-fa3ad0d24df1"). InnerVolumeSpecName "kube-api-access-mvw8d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:55:29.851063 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.851019 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/40599807-8c82-415f-a07e-fa3ad0d24df1-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:55:29.851063 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.851048 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mvw8d\" (UniqueName: \"kubernetes.io/projected/40599807-8c82-415f-a07e-fa3ad0d24df1-kube-api-access-mvw8d\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:55:29.997326 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:29.997295 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff"] Jul 06 22:55:30.000757 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:30.000699 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-b673d1-764947c6bc-fjxff"] Jul 06 22:55:30.247909 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:55:30.247874 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40599807-8c82-415f-a07e-fa3ad0d24df1" path="/var/lib/kubelet/pods/40599807-8c82-415f-a07e-fa3ad0d24df1/volumes" Jul 06 22:56:12.333109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.332975 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx"] Jul 06 22:56:12.333700 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.333334 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" podUID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" containerName="maas-api" containerID="cri-o://6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733" gracePeriod=30 Jul 06 22:56:12.381990 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.381950 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" podUID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.47:8443/health\": dial tcp 10.134.0.47:8443: connect: connection refused" Jul 06 22:56:12.591563 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.591512 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:56:12.673055 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.673019 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gkvj\" (UniqueName: \"kubernetes.io/projected/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-kube-api-access-4gkvj\") pod \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " Jul 06 22:56:12.673260 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.673141 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls\") pod \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " Jul 06 22:56:12.673260 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.673172 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-openshift-service-ca\") pod \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\" (UID: \"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f\") " Jul 06 22:56:12.673558 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.673532 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" (UID: "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:56:12.675278 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.675254 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-kube-api-access-4gkvj" (OuterVolumeSpecName: "kube-api-access-4gkvj") pod "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" (UID: "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f"). InnerVolumeSpecName "kube-api-access-4gkvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:56:12.675466 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.675439 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" (UID: "5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:56:12.774536 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.774503 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:12.774536 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.774533 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:12.774536 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.774543 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gkvj\" (UniqueName: \"kubernetes.io/projected/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f-kube-api-access-4gkvj\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:12.895641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.895553 2572 generic.go:356] "Generic (PLEG): container finished" podID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" containerID="6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733" exitCode=0 Jul 06 22:56:12.895641 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.895595 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" Jul 06 22:56:12.895850 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.895636 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" event={"ID":"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f","Type":"ContainerDied","Data":"6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733"} Jul 06 22:56:12.895850 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.895684 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx" event={"ID":"5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f","Type":"ContainerDied","Data":"97cd0c085689d36dd2de9fe5512c4f40ac6cecba0fb6fdafd3d6dd8c4c6e3a24"} Jul 06 22:56:12.895850 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.895706 2572 scope.go:117] "RemoveContainer" containerID="6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733" Jul 06 22:56:12.908136 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.908114 2572 scope.go:117] "RemoveContainer" containerID="6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733" Jul 06 22:56:12.908407 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:12.908389 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733\": container with ID starting with 6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733 not found: ID does not exist" containerID="6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733" Jul 06 22:56:12.908504 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.908423 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733"} err="failed to get container status \"6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733\": rpc error: code = NotFound desc = could not find container \"6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733\": container with ID starting with 6f5daadf8d01439883c748ab47691b730c89baa8dfecf97200a03ef3b858e733 not found: ID does not exist" Jul 06 22:56:12.919044 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.919020 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx"] Jul 06 22:56:12.922308 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:12.922290 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-f49549-987c84cbd-5jssx"] Jul 06 22:56:14.249126 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:14.249095 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" path="/var/lib/kubelet/pods/5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f/volumes" Jul 06 22:56:22.369285 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369220 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4"] Jul 06 22:56:22.369694 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369680 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40599807-8c82-415f-a07e-fa3ad0d24df1" containerName="maas-api" Jul 06 22:56:22.369742 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369696 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="40599807-8c82-415f-a07e-fa3ad0d24df1" containerName="maas-api" Jul 06 22:56:22.369742 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369711 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" containerName="maas-api" Jul 06 22:56:22.369742 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369718 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" containerName="maas-api" Jul 06 22:56:22.369834 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369782 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dbe1ca0-e18f-428f-b66d-c4753e6fbb9f" containerName="maas-api" Jul 06 22:56:22.369834 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.369795 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="40599807-8c82-415f-a07e-fa3ad0d24df1" containerName="maas-api" Jul 06 22:56:22.374221 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.374204 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.378261 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.378219 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-b793fb-serving-cert\"" Jul 06 22:56:22.410538 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.410507 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4"] Jul 06 22:56:22.466433 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.466407 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.466603 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.466488 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7zpv\" (UniqueName: \"kubernetes.io/projected/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-kube-api-access-t7zpv\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.466603 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.466522 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-openshift-service-ca\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.567655 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.567617 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.567856 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.567672 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7zpv\" (UniqueName: \"kubernetes.io/projected/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-kube-api-access-t7zpv\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.567856 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.567692 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-openshift-service-ca\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.567856 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:22.567801 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-b793fb-serving-cert: secret "maas-api-e2e-api-a-b793fb-serving-cert" not found Jul 06 22:56:22.568034 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:22.567895 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls podName:0cbce6ed-373d-4d91-8b5b-5de716a4c34d nodeName:}" failed. No retries permitted until 2026-07-06 22:56:23.067873055 +0000 UTC m=+1915.372041694 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls") pod "maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" (UID: "0cbce6ed-373d-4d91-8b5b-5de716a4c34d") : secret "maas-api-e2e-api-a-b793fb-serving-cert" not found Jul 06 22:56:22.568394 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.568378 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-openshift-service-ca\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:22.578610 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:22.578589 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7zpv\" (UniqueName: \"kubernetes.io/projected/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-kube-api-access-t7zpv\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:23.072157 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:23.072117 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:23.072368 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:23.072281 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-b793fb-serving-cert: secret "maas-api-e2e-api-a-b793fb-serving-cert" not found Jul 06 22:56:23.072368 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:23.072355 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls podName:0cbce6ed-373d-4d91-8b5b-5de716a4c34d nodeName:}" failed. No retries permitted until 2026-07-06 22:56:24.072339928 +0000 UTC m=+1916.376508566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls") pod "maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" (UID: "0cbce6ed-373d-4d91-8b5b-5de716a4c34d") : secret "maas-api-e2e-api-a-b793fb-serving-cert" not found Jul 06 22:56:24.082292 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.082225 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:24.084856 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.084830 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") pod \"maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:24.189855 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.189816 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:24.331212 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.331185 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4"] Jul 06 22:56:24.333072 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:56:24.333023 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbce6ed_373d_4d91_8b5b_5de716a4c34d.slice/crio-8f833624b39f1c41eaf357c4990030feb8a897629f025df6c372acc5e9769269 WatchSource:0}: Error finding container 8f833624b39f1c41eaf357c4990030feb8a897629f025df6c372acc5e9769269: Status 404 returned error can't find the container with id 8f833624b39f1c41eaf357c4990030feb8a897629f025df6c372acc5e9769269 Jul 06 22:56:24.335004 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.334983 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:56:24.959720 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.959685 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" event={"ID":"0cbce6ed-373d-4d91-8b5b-5de716a4c34d","Type":"ContainerStarted","Data":"be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171"} Jul 06 22:56:24.959913 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.959747 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" event={"ID":"0cbce6ed-373d-4d91-8b5b-5de716a4c34d","Type":"ContainerStarted","Data":"8f833624b39f1c41eaf357c4990030feb8a897629f025df6c372acc5e9769269"} Jul 06 22:56:24.959913 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.959780 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:24.977681 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:24.977574 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" podStartSLOduration=2.600071677 podStartE2EDuration="2.977557774s" totalImagesPullingTime="377.486097ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:24.976990203 +0000 UTC m=+1917.281158863" watchObservedRunningTime="2026-07-06 22:56:24.977557774 +0000 UTC m=+1917.281726438" Jul 06 22:56:30.461418 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.461384 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9"] Jul 06 22:56:30.465804 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.465782 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.469220 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.469201 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-e97391-serving-cert\"" Jul 06 22:56:30.486082 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.486059 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9"] Jul 06 22:56:30.541177 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.541146 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.541346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.541193 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd6sc\" (UniqueName: \"kubernetes.io/projected/0d46ab1d-da74-4fa1-862a-bc714b1de111-kube-api-access-dd6sc\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.541346 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.541291 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d46ab1d-da74-4fa1-862a-bc714b1de111-openshift-service-ca\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.642738 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.642705 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d46ab1d-da74-4fa1-862a-bc714b1de111-openshift-service-ca\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.642885 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.642756 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.642885 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.642785 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd6sc\" (UniqueName: \"kubernetes.io/projected/0d46ab1d-da74-4fa1-862a-bc714b1de111-kube-api-access-dd6sc\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.642961 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:30.642910 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-e97391-serving-cert: secret "maas-api-e2e-api-b-e97391-serving-cert" not found Jul 06 22:56:30.643006 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:30.642971 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls podName:0d46ab1d-da74-4fa1-862a-bc714b1de111 nodeName:}" failed. No retries permitted until 2026-07-06 22:56:31.142954756 +0000 UTC m=+1923.447123393 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls") pod "maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" (UID: "0d46ab1d-da74-4fa1-862a-bc714b1de111") : secret "maas-api-e2e-api-b-e97391-serving-cert" not found Jul 06 22:56:30.643355 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.643338 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d46ab1d-da74-4fa1-862a-bc714b1de111-openshift-service-ca\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.654448 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.654428 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd6sc\" (UniqueName: \"kubernetes.io/projected/0d46ab1d-da74-4fa1-862a-bc714b1de111-kube-api-access-dd6sc\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:30.971313 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:30.971282 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:31.147498 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:31.147468 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:31.147670 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:31.147619 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-b-e97391-serving-cert: secret "maas-api-e2e-api-b-e97391-serving-cert" not found Jul 06 22:56:31.147713 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:31.147680 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls podName:0d46ab1d-da74-4fa1-862a-bc714b1de111 nodeName:}" failed. No retries permitted until 2026-07-06 22:56:32.147663442 +0000 UTC m=+1924.451832079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls") pod "maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" (UID: "0d46ab1d-da74-4fa1-862a-bc714b1de111") : secret "maas-api-e2e-api-b-e97391-serving-cert" not found Jul 06 22:56:32.158054 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:32.158012 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:32.160503 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:32.160477 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") pod \"maas-api-e2e-api-b-e97391-7469957f8d-5h4s9\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:32.276371 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:32.276331 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:32.414899 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:32.414873 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9"] Jul 06 22:56:32.416832 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:56:32.416804 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d46ab1d_da74_4fa1_862a_bc714b1de111.slice/crio-520419b85319a171aa1ac36123deb2a2b898004adc948e2984c5b06b5e7a7f83 WatchSource:0}: Error finding container 520419b85319a171aa1ac36123deb2a2b898004adc948e2984c5b06b5e7a7f83: Status 404 returned error can't find the container with id 520419b85319a171aa1ac36123deb2a2b898004adc948e2984c5b06b5e7a7f83 Jul 06 22:56:33.002136 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:33.002043 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" event={"ID":"0d46ab1d-da74-4fa1-862a-bc714b1de111","Type":"ContainerStarted","Data":"7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95"} Jul 06 22:56:33.002136 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:33.002118 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:33.002335 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:33.002138 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" event={"ID":"0d46ab1d-da74-4fa1-862a-bc714b1de111","Type":"ContainerStarted","Data":"520419b85319a171aa1ac36123deb2a2b898004adc948e2984c5b06b5e7a7f83"} Jul 06 22:56:33.019199 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:33.019153 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" podStartSLOduration=2.737539209 podStartE2EDuration="3.019139017s" totalImagesPullingTime="281.599808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:56:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:56:33.017108379 +0000 UTC m=+1925.321277039" watchObservedRunningTime="2026-07-06 22:56:33.019139017 +0000 UTC m=+1925.323307675" Jul 06 22:56:39.012813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:39.012784 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:43.625028 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.624993 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4"] Jul 06 22:56:43.625614 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.625283 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" podUID="0cbce6ed-373d-4d91-8b5b-5de716a4c34d" containerName="maas-api" containerID="cri-o://be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171" gracePeriod=30 Jul 06 22:56:43.874701 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.874671 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:43.968939 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.968906 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") pod \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " Jul 06 22:56:43.969105 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.968959 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7zpv\" (UniqueName: \"kubernetes.io/projected/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-kube-api-access-t7zpv\") pod \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " Jul 06 22:56:43.969105 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.969087 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-openshift-service-ca\") pod \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\" (UID: \"0cbce6ed-373d-4d91-8b5b-5de716a4c34d\") " Jul 06 22:56:43.969453 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.969428 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0cbce6ed-373d-4d91-8b5b-5de716a4c34d" (UID: "0cbce6ed-373d-4d91-8b5b-5de716a4c34d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:56:43.971149 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.971127 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0cbce6ed-373d-4d91-8b5b-5de716a4c34d" (UID: "0cbce6ed-373d-4d91-8b5b-5de716a4c34d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:56:43.971319 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:43.971293 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-kube-api-access-t7zpv" (OuterVolumeSpecName: "kube-api-access-t7zpv") pod "0cbce6ed-373d-4d91-8b5b-5de716a4c34d" (UID: "0cbce6ed-373d-4d91-8b5b-5de716a4c34d"). InnerVolumeSpecName "kube-api-access-t7zpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:56:44.057008 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.056970 2572 generic.go:356] "Generic (PLEG): container finished" podID="0cbce6ed-373d-4d91-8b5b-5de716a4c34d" containerID="be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171" exitCode=0 Jul 06 22:56:44.057173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.057020 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" Jul 06 22:56:44.057173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.057058 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" event={"ID":"0cbce6ed-373d-4d91-8b5b-5de716a4c34d","Type":"ContainerDied","Data":"be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171"} Jul 06 22:56:44.057173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.057120 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4" event={"ID":"0cbce6ed-373d-4d91-8b5b-5de716a4c34d","Type":"ContainerDied","Data":"8f833624b39f1c41eaf357c4990030feb8a897629f025df6c372acc5e9769269"} Jul 06 22:56:44.057173 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.057143 2572 scope.go:117] "RemoveContainer" containerID="be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171" Jul 06 22:56:44.068605 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.068570 2572 scope.go:117] "RemoveContainer" containerID="be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171" Jul 06 22:56:44.069049 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:44.069028 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171\": container with ID starting with be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171 not found: ID does not exist" containerID="be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171" Jul 06 22:56:44.069135 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.069059 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171"} err="failed to get container status \"be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171\": rpc error: code = NotFound desc = could not find container \"be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171\": container with ID starting with be564ff68881454704bbf6c56f72a37d4d04aff79647b499682bc34b34881171 not found: ID does not exist" Jul 06 22:56:44.070500 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.070478 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7zpv\" (UniqueName: \"kubernetes.io/projected/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-kube-api-access-t7zpv\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:44.070500 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.070501 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:44.070634 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.070512 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0cbce6ed-373d-4d91-8b5b-5de716a4c34d-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:44.079019 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.078997 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4"] Jul 06 22:56:44.084529 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.084505 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-b793fb-546d999c6b-8p8g4"] Jul 06 22:56:44.248486 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:44.248386 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cbce6ed-373d-4d91-8b5b-5de716a4c34d" path="/var/lib/kubelet/pods/0cbce6ed-373d-4d91-8b5b-5de716a4c34d/volumes" Jul 06 22:56:50.323506 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.323462 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9"] Jul 06 22:56:50.324026 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.323705 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" podUID="0d46ab1d-da74-4fa1-862a-bc714b1de111" containerName="maas-api" containerID="cri-o://7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95" gracePeriod=30 Jul 06 22:56:50.573572 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.573514 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:50.732248 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.732203 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") pod \"0d46ab1d-da74-4fa1-862a-bc714b1de111\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " Jul 06 22:56:50.732398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.732331 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd6sc\" (UniqueName: \"kubernetes.io/projected/0d46ab1d-da74-4fa1-862a-bc714b1de111-kube-api-access-dd6sc\") pod \"0d46ab1d-da74-4fa1-862a-bc714b1de111\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " Jul 06 22:56:50.732398 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.732374 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d46ab1d-da74-4fa1-862a-bc714b1de111-openshift-service-ca\") pod \"0d46ab1d-da74-4fa1-862a-bc714b1de111\" (UID: \"0d46ab1d-da74-4fa1-862a-bc714b1de111\") " Jul 06 22:56:50.732708 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.732682 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d46ab1d-da74-4fa1-862a-bc714b1de111-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0d46ab1d-da74-4fa1-862a-bc714b1de111" (UID: "0d46ab1d-da74-4fa1-862a-bc714b1de111"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:56:50.734517 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.734489 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d46ab1d-da74-4fa1-862a-bc714b1de111-kube-api-access-dd6sc" (OuterVolumeSpecName: "kube-api-access-dd6sc") pod "0d46ab1d-da74-4fa1-862a-bc714b1de111" (UID: "0d46ab1d-da74-4fa1-862a-bc714b1de111"). InnerVolumeSpecName "kube-api-access-dd6sc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:56:50.734623 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.734540 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0d46ab1d-da74-4fa1-862a-bc714b1de111" (UID: "0d46ab1d-da74-4fa1-862a-bc714b1de111"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:56:50.833734 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.833650 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd6sc\" (UniqueName: \"kubernetes.io/projected/0d46ab1d-da74-4fa1-862a-bc714b1de111-kube-api-access-dd6sc\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:50.833734 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.833681 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d46ab1d-da74-4fa1-862a-bc714b1de111-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:50.833734 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:50.833694 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d46ab1d-da74-4fa1-862a-bc714b1de111-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 22:56:51.095614 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.095536 2572 generic.go:356] "Generic (PLEG): container finished" podID="0d46ab1d-da74-4fa1-862a-bc714b1de111" containerID="7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95" exitCode=0 Jul 06 22:56:51.095614 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.095577 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" event={"ID":"0d46ab1d-da74-4fa1-862a-bc714b1de111","Type":"ContainerDied","Data":"7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95"} Jul 06 22:56:51.095614 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.095584 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" Jul 06 22:56:51.095614 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.095613 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9" event={"ID":"0d46ab1d-da74-4fa1-862a-bc714b1de111","Type":"ContainerDied","Data":"520419b85319a171aa1ac36123deb2a2b898004adc948e2984c5b06b5e7a7f83"} Jul 06 22:56:51.095887 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.095635 2572 scope.go:117] "RemoveContainer" containerID="7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95" Jul 06 22:56:51.104813 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.104794 2572 scope.go:117] "RemoveContainer" containerID="7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95" Jul 06 22:56:51.105062 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:56:51.105044 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95\": container with ID starting with 7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95 not found: ID does not exist" containerID="7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95" Jul 06 22:56:51.105123 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.105072 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95"} err="failed to get container status \"7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95\": rpc error: code = NotFound desc = could not find container \"7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95\": container with ID starting with 7605174964dba069e9652aa20554f0887d58ac1290408289bc590968a7cf0b95 not found: ID does not exist" Jul 06 22:56:51.118977 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.118947 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9"] Jul 06 22:56:51.122783 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:51.122760 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-e97391-7469957f8d-5h4s9"] Jul 06 22:56:52.248759 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:56:52.248730 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d46ab1d-da74-4fa1-862a-bc714b1de111" path="/var/lib/kubelet/pods/0d46ab1d-da74-4fa1-862a-bc714b1de111/volumes" Jul 06 22:57:00.566761 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.566727 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7"] Jul 06 22:57:00.567410 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.567388 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d46ab1d-da74-4fa1-862a-bc714b1de111" containerName="maas-api" Jul 06 22:57:00.567410 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.567412 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d46ab1d-da74-4fa1-862a-bc714b1de111" containerName="maas-api" Jul 06 22:57:00.567578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.567436 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cbce6ed-373d-4d91-8b5b-5de716a4c34d" containerName="maas-api" Jul 06 22:57:00.567578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.567443 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbce6ed-373d-4d91-8b5b-5de716a4c34d" containerName="maas-api" Jul 06 22:57:00.567578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.567535 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cbce6ed-373d-4d91-8b5b-5de716a4c34d" containerName="maas-api" Jul 06 22:57:00.567578 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.567550 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d46ab1d-da74-4fa1-862a-bc714b1de111" containerName="maas-api" Jul 06 22:57:00.570661 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.570641 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.574322 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.574302 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-936b5d-serving-cert\"" Jul 06 22:57:00.583822 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.583799 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7"] Jul 06 22:57:00.624319 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.624283 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.624477 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.624349 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.624477 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.624422 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkwsf\" (UniqueName: \"kubernetes.io/projected/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-kube-api-access-gkwsf\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.725592 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.725561 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.725751 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.725626 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkwsf\" (UniqueName: \"kubernetes.io/projected/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-kube-api-access-gkwsf\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.725751 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.725661 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.725923 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:57:00.725767 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-936b5d-serving-cert: secret "maas-api-e2e-shared-a-936b5d-serving-cert" not found Jul 06 22:57:00.725923 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:57:00.725818 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls podName:83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc nodeName:}" failed. No retries permitted until 2026-07-06 22:57:01.225804024 +0000 UTC m=+1953.529972662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls") pod "maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" (UID: "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc") : secret "maas-api-e2e-shared-a-936b5d-serving-cert" not found Jul 06 22:57:00.726333 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.726310 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-openshift-service-ca\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:00.736660 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:00.736631 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkwsf\" (UniqueName: \"kubernetes.io/projected/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-kube-api-access-gkwsf\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:01.230683 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:01.230645 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:01.233168 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:01.233148 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls\") pod \"maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:01.482067 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:01.481975 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:01.625816 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:01.625794 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7"] Jul 06 22:57:01.628163 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:57:01.628135 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c3bd8a_99ce_4ed0_8804_26bb0aca9fbc.slice/crio-4a9dca3613fe5846ddbd63d46c29eae73bcb4d82ef381dd2957025f296cfed07 WatchSource:0}: Error finding container 4a9dca3613fe5846ddbd63d46c29eae73bcb4d82ef381dd2957025f296cfed07: Status 404 returned error can't find the container with id 4a9dca3613fe5846ddbd63d46c29eae73bcb4d82ef381dd2957025f296cfed07 Jul 06 22:57:02.162282 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:02.162226 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" event={"ID":"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc","Type":"ContainerStarted","Data":"e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269"} Jul 06 22:57:02.162478 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:02.162304 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" event={"ID":"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc","Type":"ContainerStarted","Data":"4a9dca3613fe5846ddbd63d46c29eae73bcb4d82ef381dd2957025f296cfed07"} Jul 06 22:57:02.162478 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:02.162429 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:02.180619 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:02.180506 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" podStartSLOduration=1.884102406 podStartE2EDuration="2.180489728s" totalImagesPullingTime="296.387322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:57:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:57:02.178255204 +0000 UTC m=+1954.482423860" watchObservedRunningTime="2026-07-06 22:57:02.180489728 +0000 UTC m=+1954.484658387" Jul 06 22:57:08.172442 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.172412 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 22:57:08.576535 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.576501 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln"] Jul 06 22:57:08.580584 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.580562 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.583368 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.583340 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-cf2b54-serving-cert\"" Jul 06 22:57:08.593306 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.593271 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln"] Jul 06 22:57:08.706007 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.705978 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdf5g\" (UniqueName: \"kubernetes.io/projected/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-kube-api-access-xdf5g\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.706164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.706063 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.706164 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.706086 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.806653 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.806623 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdf5g\" (UniqueName: \"kubernetes.io/projected/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-kube-api-access-xdf5g\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.806810 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.806719 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.806810 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.806744 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.806880 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:57:08.806858 2572 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-cf2b54-serving-cert: secret "maas-api-e2e-shared-b-cf2b54-serving-cert" not found Jul 06 22:57:08.806929 ip-10-0-132-23 kubenswrapper[2572]: E0706 22:57:08.806918 2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls podName:ca3ce671-88d8-4f70-9a15-2a1bb52e51f3 nodeName:}" failed. No retries permitted until 2026-07-06 22:57:09.306902661 +0000 UTC m=+1961.611071299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls") pod "maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" (UID: "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3") : secret "maas-api-e2e-shared-b-cf2b54-serving-cert" not found Jul 06 22:57:08.807447 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.807430 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:08.817647 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:08.817624 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdf5g\" (UniqueName: \"kubernetes.io/projected/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-kube-api-access-xdf5g\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:09.312109 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:09.312068 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:09.314937 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:09.314907 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls\") pod \"maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:09.494591 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:09.494555 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:09.651517 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:09.651494 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln"] Jul 06 22:57:09.654620 ip-10-0-132-23 kubenswrapper[2572]: W0706 22:57:09.654589 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca3ce671_88d8_4f70_9a15_2a1bb52e51f3.slice/crio-89e686e58edfbdbe39c9bf000f2ab2293d2bc0ebdf1b6b5503b65eee5353db0f WatchSource:0}: Error finding container 89e686e58edfbdbe39c9bf000f2ab2293d2bc0ebdf1b6b5503b65eee5353db0f: Status 404 returned error can't find the container with id 89e686e58edfbdbe39c9bf000f2ab2293d2bc0ebdf1b6b5503b65eee5353db0f Jul 06 22:57:10.205824 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:10.205781 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" event={"ID":"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3","Type":"ContainerStarted","Data":"34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf"} Jul 06 22:57:10.205980 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:10.205841 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" event={"ID":"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3","Type":"ContainerStarted","Data":"89e686e58edfbdbe39c9bf000f2ab2293d2bc0ebdf1b6b5503b65eee5353db0f"} Jul 06 22:57:10.205980 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:10.205922 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:57:10.228124 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:10.228075 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" podStartSLOduration=1.845127396 podStartE2EDuration="2.228060265s" totalImagesPullingTime="382.932869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:57:10.226295783 +0000 UTC m=+1962.530464441" watchObservedRunningTime="2026-07-06 22:57:10.228060265 +0000 UTC m=+1962.532228925" Jul 06 22:57:16.216343 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:57:16.216318 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 22:59:28.279197 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:59:28.279165 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 22:59:28.281711 ip-10-0-132-23 kubenswrapper[2572]: I0706 22:59:28.281329 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 23:00:00.138935 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.138894 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722980-cbcnx"] Jul 06 23:00:00.143083 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.143057 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m"] Jul 06 23:00:00.143260 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.143174 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:00.146547 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.146528 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7"] Jul 06 23:00:00.146681 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.146661 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:00.150054 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.150036 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:00.158731 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.158696 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722980-cbcnx"] Jul 06 23:00:00.162295 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.162270 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7"] Jul 06 23:00:00.164893 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.164871 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m"] Jul 06 23:00:00.253851 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.253812 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5pf4\" (UniqueName: \"kubernetes.io/projected/f8f50348-004a-4d7d-970c-dadf4c647190-kube-api-access-p5pf4\") pod \"maas-api-key-cleanup-29722980-cbcnx\" (UID: \"f8f50348-004a-4d7d-970c-dadf4c647190\") " pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:00.254027 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.253946 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjchv\" (UniqueName: \"kubernetes.io/projected/c1211d47-79c1-46a5-a1d2-111a9b5c1dca-kube-api-access-pjchv\") pod \"maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m\" (UID: \"c1211d47-79c1-46a5-a1d2-111a9b5c1dca\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:00.254027 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.253976 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wls6r\" (UniqueName: \"kubernetes.io/projected/7d2e55be-aa25-40af-bf45-1dde0e498373-kube-api-access-wls6r\") pod \"maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7\" (UID: \"7d2e55be-aa25-40af-bf45-1dde0e498373\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:00.355306 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.355265 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjchv\" (UniqueName: \"kubernetes.io/projected/c1211d47-79c1-46a5-a1d2-111a9b5c1dca-kube-api-access-pjchv\") pod \"maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m\" (UID: \"c1211d47-79c1-46a5-a1d2-111a9b5c1dca\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:00.355573 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.355551 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wls6r\" (UniqueName: \"kubernetes.io/projected/7d2e55be-aa25-40af-bf45-1dde0e498373-kube-api-access-wls6r\") pod \"maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7\" (UID: \"7d2e55be-aa25-40af-bf45-1dde0e498373\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:00.355778 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.355763 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5pf4\" (UniqueName: \"kubernetes.io/projected/f8f50348-004a-4d7d-970c-dadf4c647190-kube-api-access-p5pf4\") pod \"maas-api-key-cleanup-29722980-cbcnx\" (UID: \"f8f50348-004a-4d7d-970c-dadf4c647190\") " pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:00.364830 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.364803 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wls6r\" (UniqueName: \"kubernetes.io/projected/7d2e55be-aa25-40af-bf45-1dde0e498373-kube-api-access-wls6r\") pod \"maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7\" (UID: \"7d2e55be-aa25-40af-bf45-1dde0e498373\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:00.364967 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.364807 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5pf4\" (UniqueName: \"kubernetes.io/projected/f8f50348-004a-4d7d-970c-dadf4c647190-kube-api-access-p5pf4\") pod \"maas-api-key-cleanup-29722980-cbcnx\" (UID: \"f8f50348-004a-4d7d-970c-dadf4c647190\") " pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:00.364967 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.364867 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjchv\" (UniqueName: \"kubernetes.io/projected/c1211d47-79c1-46a5-a1d2-111a9b5c1dca-kube-api-access-pjchv\") pod \"maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m\" (UID: \"c1211d47-79c1-46a5-a1d2-111a9b5c1dca\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:00.455367 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.455329 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:00.464586 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.464563 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:00.472870 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.472842 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:00.615594 ip-10-0-132-23 kubenswrapper[2572]: W0706 23:00:00.615556 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f50348_004a_4d7d_970c_dadf4c647190.slice/crio-c4fbfd0705a4e2314b4f4ddf1272dd3c1f3683c0525362cbeef6ff53fd5d1d2c WatchSource:0}: Error finding container c4fbfd0705a4e2314b4f4ddf1272dd3c1f3683c0525362cbeef6ff53fd5d1d2c: Status 404 returned error can't find the container with id c4fbfd0705a4e2314b4f4ddf1272dd3c1f3683c0525362cbeef6ff53fd5d1d2c Jul 06 23:00:00.617804 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.617584 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722980-cbcnx"] Jul 06 23:00:00.854855 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.854822 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m"] Jul 06 23:00:00.856343 ip-10-0-132-23 kubenswrapper[2572]: W0706 23:00:00.856309 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1211d47_79c1_46a5_a1d2_111a9b5c1dca.slice/crio-c8f3a6fe1b8b39d3cf3a3aaf106350b20f134533f868d0b6fe818e8102e55349 WatchSource:0}: Error finding container c8f3a6fe1b8b39d3cf3a3aaf106350b20f134533f868d0b6fe818e8102e55349: Status 404 returned error can't find the container with id c8f3a6fe1b8b39d3cf3a3aaf106350b20f134533f868d0b6fe818e8102e55349 Jul 06 23:00:00.857022 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:00.856991 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7"] Jul 06 23:00:00.858144 ip-10-0-132-23 kubenswrapper[2572]: W0706 23:00:00.858119 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d2e55be_aa25_40af_bf45_1dde0e498373.slice/crio-a25277b72fd601c9ee5442b0fe0ee079572ca22c65a0f1daf8b78bffd17db9c0 WatchSource:0}: Error finding container a25277b72fd601c9ee5442b0fe0ee079572ca22c65a0f1daf8b78bffd17db9c0: Status 404 returned error can't find the container with id a25277b72fd601c9ee5442b0fe0ee079572ca22c65a0f1daf8b78bffd17db9c0 Jul 06 23:00:01.058698 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:01.058611 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" event={"ID":"f8f50348-004a-4d7d-970c-dadf4c647190","Type":"ContainerStarted","Data":"205f36e0f5ccab8b3ae6496c6a66e25d989db426f7d2e4fa641c453b69a8ddbf"} Jul 06 23:00:01.058698 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:01.058673 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" event={"ID":"f8f50348-004a-4d7d-970c-dadf4c647190","Type":"ContainerStarted","Data":"c4fbfd0705a4e2314b4f4ddf1272dd3c1f3683c0525362cbeef6ff53fd5d1d2c"} Jul 06 23:00:01.059807 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:01.059779 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" event={"ID":"c1211d47-79c1-46a5-a1d2-111a9b5c1dca","Type":"ContainerStarted","Data":"c8f3a6fe1b8b39d3cf3a3aaf106350b20f134533f868d0b6fe818e8102e55349"} Jul 06 23:00:01.060839 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:01.060819 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" event={"ID":"7d2e55be-aa25-40af-bf45-1dde0e498373","Type":"ContainerStarted","Data":"a25277b72fd601c9ee5442b0fe0ee079572ca22c65a0f1daf8b78bffd17db9c0"} Jul 06 23:00:01.074962 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:01.074912 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" podStartSLOduration=1.074897676 podStartE2EDuration="1.074897676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:00:01.07320921 +0000 UTC m=+2133.377377870" watchObservedRunningTime="2026-07-06 23:00:01.074897676 +0000 UTC m=+2133.379066339" Jul 06 23:00:02.067167 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:02.067133 2572 generic.go:356] "Generic (PLEG): container finished" podID="7d2e55be-aa25-40af-bf45-1dde0e498373" containerID="4f3a3f916c187930ab05981a48a90b95ef014cf27bf6c85ad62f3b4c1e4f762d" exitCode=0 Jul 06 23:00:02.067571 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:02.067217 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" event={"ID":"7d2e55be-aa25-40af-bf45-1dde0e498373","Type":"ContainerDied","Data":"4f3a3f916c187930ab05981a48a90b95ef014cf27bf6c85ad62f3b4c1e4f762d"} Jul 06 23:00:02.068683 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:02.068652 2572 generic.go:356] "Generic (PLEG): container finished" podID="f8f50348-004a-4d7d-970c-dadf4c647190" containerID="205f36e0f5ccab8b3ae6496c6a66e25d989db426f7d2e4fa641c453b69a8ddbf" exitCode=0 Jul 06 23:00:02.068813 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:02.068734 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" event={"ID":"f8f50348-004a-4d7d-970c-dadf4c647190","Type":"ContainerDied","Data":"205f36e0f5ccab8b3ae6496c6a66e25d989db426f7d2e4fa641c453b69a8ddbf"} Jul 06 23:00:02.070143 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:02.070124 2572 generic.go:356] "Generic (PLEG): container finished" podID="c1211d47-79c1-46a5-a1d2-111a9b5c1dca" containerID="6e4afbfb8df8fd7f094de9b15015db6859b843c0b6ca8e12dd2b80740dad3bb0" exitCode=0 Jul 06 23:00:02.070218 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:02.070154 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" event={"ID":"c1211d47-79c1-46a5-a1d2-111a9b5c1dca","Type":"ContainerDied","Data":"6e4afbfb8df8fd7f094de9b15015db6859b843c0b6ca8e12dd2b80740dad3bb0"} Jul 06 23:00:03.249680 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.249652 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:03.253225 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.253205 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:03.280618 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.280586 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:03.385099 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.385007 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5pf4\" (UniqueName: \"kubernetes.io/projected/f8f50348-004a-4d7d-970c-dadf4c647190-kube-api-access-p5pf4\") pod \"f8f50348-004a-4d7d-970c-dadf4c647190\" (UID: \"f8f50348-004a-4d7d-970c-dadf4c647190\") " Jul 06 23:00:03.385099 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.385093 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wls6r\" (UniqueName: \"kubernetes.io/projected/7d2e55be-aa25-40af-bf45-1dde0e498373-kube-api-access-wls6r\") pod \"7d2e55be-aa25-40af-bf45-1dde0e498373\" (UID: \"7d2e55be-aa25-40af-bf45-1dde0e498373\") " Jul 06 23:00:03.385329 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.385166 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjchv\" (UniqueName: \"kubernetes.io/projected/c1211d47-79c1-46a5-a1d2-111a9b5c1dca-kube-api-access-pjchv\") pod \"c1211d47-79c1-46a5-a1d2-111a9b5c1dca\" (UID: \"c1211d47-79c1-46a5-a1d2-111a9b5c1dca\") " Jul 06 23:00:03.387458 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.387422 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f50348-004a-4d7d-970c-dadf4c647190-kube-api-access-p5pf4" (OuterVolumeSpecName: "kube-api-access-p5pf4") pod "f8f50348-004a-4d7d-970c-dadf4c647190" (UID: "f8f50348-004a-4d7d-970c-dadf4c647190"). InnerVolumeSpecName "kube-api-access-p5pf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:00:03.387458 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.387453 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1211d47-79c1-46a5-a1d2-111a9b5c1dca-kube-api-access-pjchv" (OuterVolumeSpecName: "kube-api-access-pjchv") pod "c1211d47-79c1-46a5-a1d2-111a9b5c1dca" (UID: "c1211d47-79c1-46a5-a1d2-111a9b5c1dca"). InnerVolumeSpecName "kube-api-access-pjchv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:00:03.387606 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.387429 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d2e55be-aa25-40af-bf45-1dde0e498373-kube-api-access-wls6r" (OuterVolumeSpecName: "kube-api-access-wls6r") pod "7d2e55be-aa25-40af-bf45-1dde0e498373" (UID: "7d2e55be-aa25-40af-bf45-1dde0e498373"). InnerVolumeSpecName "kube-api-access-wls6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:00:03.486014 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.485978 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjchv\" (UniqueName: \"kubernetes.io/projected/c1211d47-79c1-46a5-a1d2-111a9b5c1dca-kube-api-access-pjchv\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:00:03.486014 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.486008 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5pf4\" (UniqueName: \"kubernetes.io/projected/f8f50348-004a-4d7d-970c-dadf4c647190-kube-api-access-p5pf4\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:00:03.486014 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:03.486018 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wls6r\" (UniqueName: \"kubernetes.io/projected/7d2e55be-aa25-40af-bf45-1dde0e498373-kube-api-access-wls6r\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:00:04.082263 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.082220 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" Jul 06 23:00:04.082479 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.082264 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722980-cbcnx" event={"ID":"f8f50348-004a-4d7d-970c-dadf4c647190","Type":"ContainerDied","Data":"c4fbfd0705a4e2314b4f4ddf1272dd3c1f3683c0525362cbeef6ff53fd5d1d2c"} Jul 06 23:00:04.082479 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.082317 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4fbfd0705a4e2314b4f4ddf1272dd3c1f3683c0525362cbeef6ff53fd5d1d2c" Jul 06 23:00:04.083665 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.083647 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" Jul 06 23:00:04.083786 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.083691 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m" event={"ID":"c1211d47-79c1-46a5-a1d2-111a9b5c1dca","Type":"ContainerDied","Data":"c8f3a6fe1b8b39d3cf3a3aaf106350b20f134533f868d0b6fe818e8102e55349"} Jul 06 23:00:04.083786 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.083735 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8f3a6fe1b8b39d3cf3a3aaf106350b20f134533f868d0b6fe818e8102e55349" Jul 06 23:00:04.085125 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.085099 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" Jul 06 23:00:04.085125 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.085104 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7" event={"ID":"7d2e55be-aa25-40af-bf45-1dde0e498373","Type":"ContainerDied","Data":"a25277b72fd601c9ee5442b0fe0ee079572ca22c65a0f1daf8b78bffd17db9c0"} Jul 06 23:00:04.085296 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:00:04.085144 2572 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a25277b72fd601c9ee5442b0fe0ee079572ca22c65a0f1daf8b78bffd17db9c0" Jul 06 23:03:22.796064 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:22.796023 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7"] Jul 06 23:03:22.796697 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:22.796368 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" podUID="83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" containerName="maas-api" containerID="cri-o://e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269" gracePeriod=30 Jul 06 23:03:22.825873 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:22.825834 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m"] Jul 06 23:03:22.828062 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:22.828038 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-936b5d-29722980-62m7m"] Jul 06 23:03:23.056090 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.056028 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 23:03:23.117893 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.117859 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkwsf\" (UniqueName: \"kubernetes.io/projected/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-kube-api-access-gkwsf\") pod \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " Jul 06 23:03:23.118063 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.117953 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls\") pod \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " Jul 06 23:03:23.118138 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.118117 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-openshift-service-ca\") pod \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\" (UID: \"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc\") " Jul 06 23:03:23.118538 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.118513 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" (UID: "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:03:23.120094 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.120066 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" (UID: "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:03:23.120182 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.120164 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-kube-api-access-gkwsf" (OuterVolumeSpecName: "kube-api-access-gkwsf") pod "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" (UID: "83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc"). InnerVolumeSpecName "kube-api-access-gkwsf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:03:23.134435 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.134398 2572 generic.go:356] "Generic (PLEG): container finished" podID="83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" containerID="e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269" exitCode=0 Jul 06 23:03:23.134579 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.134451 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" Jul 06 23:03:23.134579 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.134473 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" event={"ID":"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc","Type":"ContainerDied","Data":"e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269"} Jul 06 23:03:23.134579 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.134527 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7" event={"ID":"83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc","Type":"ContainerDied","Data":"4a9dca3613fe5846ddbd63d46c29eae73bcb4d82ef381dd2957025f296cfed07"} Jul 06 23:03:23.134579 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.134551 2572 scope.go:117] "RemoveContainer" containerID="e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269" Jul 06 23:03:23.144531 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.144515 2572 scope.go:117] "RemoveContainer" containerID="e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269" Jul 06 23:03:23.144795 ip-10-0-132-23 kubenswrapper[2572]: E0706 23:03:23.144776 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269\": container with ID starting with e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269 not found: ID does not exist" containerID="e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269" Jul 06 23:03:23.144846 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.144805 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269"} err="failed to get container status \"e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269\": rpc error: code = NotFound desc = could not find container \"e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269\": container with ID starting with e6d83ada2f506d3f0a53677fba776f922dae32e3933effa039c153848a6a5269 not found: ID does not exist" Jul 06 23:03:23.157651 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.157618 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7"] Jul 06 23:03:23.159600 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.159576 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-936b5d-76c9d95845-rkdg7"] Jul 06 23:03:23.219379 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.219340 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:03:23.219379 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.219374 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkwsf\" (UniqueName: \"kubernetes.io/projected/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-kube-api-access-gkwsf\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:03:23.219379 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:23.219387 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:03:24.248506 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:24.248473 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" path="/var/lib/kubelet/pods/83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc/volumes" Jul 06 23:03:24.248886 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:03:24.248811 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1211d47-79c1-46a5-a1d2-111a9b5c1dca" path="/var/lib/kubelet/pods/c1211d47-79c1-46a5-a1d2-111a9b5c1dca/volumes" Jul 06 23:04:05.822732 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:05.822685 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln"] Jul 06 23:04:05.823377 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:05.822933 2572 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" podUID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" containerName="maas-api" containerID="cri-o://34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf" gracePeriod=30 Jul 06 23:04:05.848975 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:05.848939 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7"] Jul 06 23:04:05.852573 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:05.852547 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cf2b54-29722980-kh2x7"] Jul 06 23:04:06.249575 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.249538 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d2e55be-aa25-40af-bf45-1dde0e498373" path="/var/lib/kubelet/pods/7d2e55be-aa25-40af-bf45-1dde0e498373/volumes" Jul 06 23:04:06.284199 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.284176 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 23:04:06.342815 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.342784 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-openshift-service-ca\") pod \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " Jul 06 23:04:06.342994 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.342851 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdf5g\" (UniqueName: \"kubernetes.io/projected/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-kube-api-access-xdf5g\") pod \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " Jul 06 23:04:06.342994 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.342930 2572 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls\") pod \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\" (UID: \"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3\") " Jul 06 23:04:06.343266 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.343224 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" (UID: "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 23:04:06.345208 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.345185 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-kube-api-access-xdf5g" (OuterVolumeSpecName: "kube-api-access-xdf5g") pod "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" (UID: "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3"). InnerVolumeSpecName "kube-api-access-xdf5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 23:04:06.345208 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.345198 2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" (UID: "ca3ce671-88d8-4f70-9a15-2a1bb52e51f3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 23:04:06.361397 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.361318 2572 generic.go:356] "Generic (PLEG): container finished" podID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" containerID="34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf" exitCode=0 Jul 06 23:04:06.361397 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.361359 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" event={"ID":"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3","Type":"ContainerDied","Data":"34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf"} Jul 06 23:04:06.361397 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.361368 2572 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" Jul 06 23:04:06.361616 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.361402 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" event={"ID":"ca3ce671-88d8-4f70-9a15-2a1bb52e51f3","Type":"ContainerDied","Data":"89e686e58edfbdbe39c9bf000f2ab2293d2bc0ebdf1b6b5503b65eee5353db0f"} Jul 06 23:04:06.361616 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.361424 2572 scope.go:117] "RemoveContainer" containerID="34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf" Jul 06 23:04:06.370601 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.370583 2572 scope.go:117] "RemoveContainer" containerID="34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf" Jul 06 23:04:06.370887 ip-10-0-132-23 kubenswrapper[2572]: E0706 23:04:06.370869 2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf\": container with ID starting with 34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf not found: ID does not exist" containerID="34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf" Jul 06 23:04:06.370936 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.370898 2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf"} err="failed to get container status \"34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf\": rpc error: code = NotFound desc = could not find container \"34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf\": container with ID starting with 34dcf5eb7d890c6c83897883f2b3f71782cc4d94691f539b83aabcbe1ab9b8cf not found: ID does not exist" Jul 06 23:04:06.384178 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.384151 2572 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln"] Jul 06 23:04:06.386037 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.386017 2572 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln"] Jul 06 23:04:06.444656 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.444620 2572 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-openshift-service-ca\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:04:06.444656 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.444649 2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdf5g\" (UniqueName: \"kubernetes.io/projected/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-kube-api-access-xdf5g\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:04:06.444656 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:06.444662 2572 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3-maas-api-tls\") on node \"ip-10-0-132-23.ec2.internal\" DevicePath \"\"" Jul 06 23:04:08.248500 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:08.248464 2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" path="/var/lib/kubelet/pods/ca3ce671-88d8-4f70-9a15-2a1bb52e51f3/volumes" Jul 06 23:04:09.211712 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:09.211646 2572 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-b-cf2b54-5c75689bc-7zzln" podUID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.51:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 06 23:04:17.108817 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.108783 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-h6l2m_c9548ebb-ca06-4375-be3e-bb36a9add5de/manager/0.log" Jul 06 23:04:17.223464 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.223436 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-69b48b77f8-dvmfq_d46a7afc-865a-4459-9ae3-aaf250fc623d/maas-api/0.log" Jul 06 23:04:17.333971 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.333936 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722965-qzgzs_217511b3-53ed-4fe0-a455-e931e28582c2/cleanup/0.log" Jul 06 23:04:17.445405 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.445373 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722980-cbcnx_f8f50348-004a-4d7d-970c-dadf4c647190/cleanup/0.log" Jul 06 23:04:17.566928 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.566902 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d57c96767-mhkrw_49cfd9ff-47de-4085-ab62-ec75ed383487/manager/0.log" Jul 06 23:04:17.907623 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.907541 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84b8787b6c-gjb52_e63e4f76-619a-4a3c-9292-e8e02ec385fa/copy-manifests/0.log" Jul 06 23:04:17.914358 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:17.914333 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84b8787b6c-gjb52_e63e4f76-619a-4a3c-9292-e8e02ec385fa/manager/0.log" Jul 06 23:04:18.287523 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:18.287484 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-bz7wl_94ca973b-3a14-47ba-b1ee-c5ab3dfadb06/postgres/0.log" Jul 06 23:04:19.941284 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:19.941225 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-j4shm_3b7d760d-80cc-4f6f-b37f-163db5b39c4e/kuadrant-console-plugin/0.log" Jul 06 23:04:21.092817 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:21.092788 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6b7b4fbb-m8lxl_a00628a2-128d-4499-b556-f8ca7f43572b/kube-auth-proxy/0.log" Jul 06 23:04:21.324041 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:21.324005 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-km4lp_434b4986-8680-4a50-b734-dad514382810/payload-pre-processing/0.log" Jul 06 23:04:21.442207 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:21.442176 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-88j4z_092bfe7a-d748-44dc-80d2-31d5d215d726/payload-processing/0.log" Jul 06 23:04:21.997203 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:21.997173 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-mgqxk_8d692b86-229a-44ce-88e7-9b217716e128/main/0.log" Jul 06 23:04:22.109534 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:22.109498 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-dhhw7_d31d87b7-8bae-4f46-aab3-01dc25bf22e9/main/0.log" Jul 06 23:04:26.703802 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.703762 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-768zc/must-gather-56vss"] Jul 06 23:04:26.704226 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704211 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" containerName="maas-api" Jul 06 23:04:26.704226 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704228 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" containerName="maas-api" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704256 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d2e55be-aa25-40af-bf45-1dde0e498373" containerName="cleanup" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704262 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d2e55be-aa25-40af-bf45-1dde0e498373" containerName="cleanup" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704280 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" containerName="maas-api" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704290 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" containerName="maas-api" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704309 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1211d47-79c1-46a5-a1d2-111a9b5c1dca" containerName="cleanup" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704314 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1211d47-79c1-46a5-a1d2-111a9b5c1dca" containerName="cleanup" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704323 2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f50348-004a-4d7d-970c-dadf4c647190" containerName="cleanup" Jul 06 23:04:26.704334 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704328 2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f50348-004a-4d7d-970c-dadf4c647190" containerName="cleanup" Jul 06 23:04:26.704580 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704404 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f50348-004a-4d7d-970c-dadf4c647190" containerName="cleanup" Jul 06 23:04:26.704580 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704412 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d2e55be-aa25-40af-bf45-1dde0e498373" containerName="cleanup" Jul 06 23:04:26.704580 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704420 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="83c3bd8a-99ce-4ed0-8804-26bb0aca9fbc" containerName="maas-api" Jul 06 23:04:26.704580 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704428 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca3ce671-88d8-4f70-9a15-2a1bb52e51f3" containerName="maas-api" Jul 06 23:04:26.704580 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.704434 2572 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1211d47-79c1-46a5-a1d2-111a9b5c1dca" containerName="cleanup" Jul 06 23:04:26.709375 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.709352 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:26.712408 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.712369 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-768zc\"/\"openshift-service-ca.crt\"" Jul 06 23:04:26.712714 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.712698 2572 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-768zc\"/\"kube-root-ca.crt\"" Jul 06 23:04:26.713864 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.713850 2572 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-768zc\"/\"default-dockercfg-ks99f\"" Jul 06 23:04:26.719536 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.719511 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-768zc/must-gather-56vss"] Jul 06 23:04:26.843990 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.843943 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbjtq\" (UniqueName: \"kubernetes.io/projected/eecf2321-e2ec-41e8-b9ee-0d000296d4ad-kube-api-access-kbjtq\") pod \"must-gather-56vss\" (UID: \"eecf2321-e2ec-41e8-b9ee-0d000296d4ad\") " pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:26.844174 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.844090 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eecf2321-e2ec-41e8-b9ee-0d000296d4ad-must-gather-output\") pod \"must-gather-56vss\" (UID: \"eecf2321-e2ec-41e8-b9ee-0d000296d4ad\") " pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:26.945597 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.945557 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eecf2321-e2ec-41e8-b9ee-0d000296d4ad-must-gather-output\") pod \"must-gather-56vss\" (UID: \"eecf2321-e2ec-41e8-b9ee-0d000296d4ad\") " pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:26.945784 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.945621 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbjtq\" (UniqueName: \"kubernetes.io/projected/eecf2321-e2ec-41e8-b9ee-0d000296d4ad-kube-api-access-kbjtq\") pod \"must-gather-56vss\" (UID: \"eecf2321-e2ec-41e8-b9ee-0d000296d4ad\") " pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:26.945928 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.945909 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/eecf2321-e2ec-41e8-b9ee-0d000296d4ad-must-gather-output\") pod \"must-gather-56vss\" (UID: \"eecf2321-e2ec-41e8-b9ee-0d000296d4ad\") " pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:26.954406 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:26.954331 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbjtq\" (UniqueName: \"kubernetes.io/projected/eecf2321-e2ec-41e8-b9ee-0d000296d4ad-kube-api-access-kbjtq\") pod \"must-gather-56vss\" (UID: \"eecf2321-e2ec-41e8-b9ee-0d000296d4ad\") " pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:27.020839 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:27.020793 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-768zc/must-gather-56vss" Jul 06 23:04:27.162444 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:27.162415 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-768zc/must-gather-56vss"] Jul 06 23:04:27.164293 ip-10-0-132-23 kubenswrapper[2572]: W0706 23:04:27.164222 2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeecf2321_e2ec_41e8_b9ee_0d000296d4ad.slice/crio-d1d2a9f146e9ba5f0a97973d3a16b11625d5f7ff305b9707f6ef3f6dd2e629ae WatchSource:0}: Error finding container d1d2a9f146e9ba5f0a97973d3a16b11625d5f7ff305b9707f6ef3f6dd2e629ae: Status 404 returned error can't find the container with id d1d2a9f146e9ba5f0a97973d3a16b11625d5f7ff305b9707f6ef3f6dd2e629ae Jul 06 23:04:27.166065 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:27.166047 2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 23:04:27.475746 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:27.475711 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-768zc/must-gather-56vss" event={"ID":"eecf2321-e2ec-41e8-b9ee-0d000296d4ad","Type":"ContainerStarted","Data":"d1d2a9f146e9ba5f0a97973d3a16b11625d5f7ff305b9707f6ef3f6dd2e629ae"} Jul 06 23:04:28.338351 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:28.338210 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 23:04:28.355015 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:28.339361 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7tdp7_3c5f9cb7-f0f2-470d-811a-6a0015455d42/ovn-acl-logging/0.log" Jul 06 23:04:28.489809 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:28.489764 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-768zc/must-gather-56vss" event={"ID":"eecf2321-e2ec-41e8-b9ee-0d000296d4ad","Type":"ContainerStarted","Data":"f69fa589abd6fdd4296a5a4ec456b1ea4ce7b1977b9251946aa3ff0c796b9e53"} Jul 06 23:04:28.490003 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:28.489825 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-768zc/must-gather-56vss" event={"ID":"eecf2321-e2ec-41e8-b9ee-0d000296d4ad","Type":"ContainerStarted","Data":"24855445b636eabcb5939e1284dac80b9ceda74bc8d35a943e05dbd2a6cb0cca"} Jul 06 23:04:28.510020 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:28.509951 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-768zc/must-gather-56vss" podStartSLOduration=1.920348763 podStartE2EDuration="2.509931411s" totalImagesPullingTime="589.582648ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 23:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:04:28.505620551 +0000 UTC m=+2400.809789223" watchObservedRunningTime="2026-07-06 23:04:28.509931411 +0000 UTC m=+2400.814100073" Jul 06 23:04:29.516686 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:29.516656 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-vlvg2_8b4cf0a7-678c-4f21-8d45-a46b9e1dd522/global-pull-secret-syncer/0.log" Jul 06 23:04:29.609184 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:29.609149 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-krqkm_4f6c5d5e-2d4b-4b0b-8eae-12039897236e/konnectivity-agent/0.log" Jul 06 23:04:29.712656 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:29.712618 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-23.ec2.internal_4d8e8123e63af15df39698cf832a3600/haproxy/0.log" Jul 06 23:04:34.925024 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:34.924144 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-j4shm_3b7d760d-80cc-4f6f-b37f-163db5b39c4e/kuadrant-console-plugin/0.log" Jul 06 23:04:37.025384 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.025346 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/alertmanager/0.log" Jul 06 23:04:37.051255 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.051202 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/config-reloader/0.log" Jul 06 23:04:37.082853 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.082823 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/kube-rbac-proxy-web/0.log" Jul 06 23:04:37.110534 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.110503 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/kube-rbac-proxy/0.log" Jul 06 23:04:37.144044 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.144015 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/kube-rbac-proxy-metric/0.log" Jul 06 23:04:37.183258 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.180439 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/prom-label-proxy/0.log" Jul 06 23:04:37.212325 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.212259 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_3b2d387e-c01a-4b0f-8eea-6425ed5e8d42/init-config-reloader/0.log" Jul 06 23:04:37.297309 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.297132 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fpsmc_9454bdb1-f37d-4995-a1fc-0a668f8b351b/kube-state-metrics/0.log" Jul 06 23:04:37.315256 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.315139 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fpsmc_9454bdb1-f37d-4995-a1fc-0a668f8b351b/kube-rbac-proxy-main/0.log" Jul 06 23:04:37.339585 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.339553 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fpsmc_9454bdb1-f37d-4995-a1fc-0a668f8b351b/kube-rbac-proxy-self/0.log" Jul 06 23:04:37.367017 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.366986 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c86cd7fb9-l6rpf_d3d957cc-6cf8-4f15-a836-1f8392b79312/metrics-server/0.log" Jul 06 23:04:37.428103 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.428076 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-79sg4_5f2dfeb0-402f-46a7-8c00-71b2edad6bbb/node-exporter/0.log" Jul 06 23:04:37.450874 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.450843 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-79sg4_5f2dfeb0-402f-46a7-8c00-71b2edad6bbb/kube-rbac-proxy/0.log" Jul 06 23:04:37.472779 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.472738 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-79sg4_5f2dfeb0-402f-46a7-8c00-71b2edad6bbb/init-textfile/0.log" Jul 06 23:04:37.678303 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.678264 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-zl6bp_a0cb8137-ca52-442f-b047-6fc455e12f2f/kube-rbac-proxy-main/0.log" Jul 06 23:04:37.699609 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.699581 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-zl6bp_a0cb8137-ca52-442f-b047-6fc455e12f2f/kube-rbac-proxy-self/0.log" Jul 06 23:04:37.721904 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.721874 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-zl6bp_a0cb8137-ca52-442f-b047-6fc455e12f2f/openshift-state-metrics/0.log" Jul 06 23:04:37.764757 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.764723 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/prometheus/0.log" Jul 06 23:04:37.787660 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.787627 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/config-reloader/0.log" Jul 06 23:04:37.813065 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.812977 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/thanos-sidecar/0.log" Jul 06 23:04:37.839431 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.839388 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/kube-rbac-proxy-web/0.log" Jul 06 23:04:37.865365 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.865333 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/kube-rbac-proxy/0.log" Jul 06 23:04:37.893509 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.893480 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/kube-rbac-proxy-thanos/0.log" Jul 06 23:04:37.921957 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.921925 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f96da12e-6cde-49ae-b56d-1b01754d3289/init-config-reloader/0.log" Jul 06 23:04:37.951531 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.951497 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5fnll_80072443-7f2e-4c71-b73c-6f63230582ae/prometheus-operator/0.log" Jul 06 23:04:37.978738 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:37.978712 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-5fnll_80072443-7f2e-4c71-b73c-6f63230582ae/kube-rbac-proxy/0.log" Jul 06 23:04:38.011176 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.011145 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-46ktv_11dd52dc-1153-4a2c-9a96-55b262af17ba/prometheus-operator-admission-webhook/0.log" Jul 06 23:04:38.052709 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.052670 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b6c44fc4d-fvlwc_188689f4-7e55-4876-87bc-0bc264dc6205/telemeter-client/0.log" Jul 06 23:04:38.071422 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.071341 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b6c44fc4d-fvlwc_188689f4-7e55-4876-87bc-0bc264dc6205/reload/0.log" Jul 06 23:04:38.095558 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.095525 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6b6c44fc4d-fvlwc_188689f4-7e55-4876-87bc-0bc264dc6205/kube-rbac-proxy/0.log" Jul 06 23:04:38.129101 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.129073 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c684f54f5-bssqq_d77792fd-44c9-4ebb-ab23-0e690f960164/thanos-query/0.log" Jul 06 23:04:38.153191 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.153125 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c684f54f5-bssqq_d77792fd-44c9-4ebb-ab23-0e690f960164/kube-rbac-proxy-web/0.log" Jul 06 23:04:38.177470 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.177443 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c684f54f5-bssqq_d77792fd-44c9-4ebb-ab23-0e690f960164/kube-rbac-proxy/0.log" Jul 06 23:04:38.201005 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.200970 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c684f54f5-bssqq_d77792fd-44c9-4ebb-ab23-0e690f960164/prom-label-proxy/0.log" Jul 06 23:04:38.224768 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.224733 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c684f54f5-bssqq_d77792fd-44c9-4ebb-ab23-0e690f960164/kube-rbac-proxy-rules/0.log" Jul 06 23:04:38.247168 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:38.247143 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7c684f54f5-bssqq_d77792fd-44c9-4ebb-ab23-0e690f960164/kube-rbac-proxy-metrics/0.log" Jul 06 23:04:40.253288 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.253254 2572 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z"] Jul 06 23:04:40.262661 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.261968 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.266338 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.266304 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z"] Jul 06 23:04:40.326908 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.326853 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-proc\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.327105 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.326939 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-sys\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.327105 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.326999 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-podres\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.327105 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.327062 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5lj\" (UniqueName: \"kubernetes.io/projected/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-kube-api-access-mp5lj\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.327105 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.327090 2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-lib-modules\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428443 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428387 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-proc\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428638 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428461 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-sys\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428638 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428515 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-podres\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428638 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428558 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5lj\" (UniqueName: \"kubernetes.io/projected/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-kube-api-access-mp5lj\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428638 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428584 2572 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-lib-modules\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428855 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428765 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-lib-modules\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428855 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428826 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-proc\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.428959 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428886 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-sys\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.429012 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.428955 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-podres\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.438903 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.438870 2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5lj\" (UniqueName: \"kubernetes.io/projected/9295f36e-f2a8-4e63-95b6-1fd6fe05c731-kube-api-access-mp5lj\") pod \"perf-node-gather-daemonset-bv22z\" (UID: \"9295f36e-f2a8-4e63-95b6-1fd6fe05c731\") " pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.578622 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.578524 2572 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:40.733112 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.731295 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5dd4b778fb-5x8c6_af8a0f70-3396-4dd8-a24d-67dc2e2e6039/console/0.log" Jul 06 23:04:40.771313 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:40.771281 2572 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z"] Jul 06 23:04:41.601422 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:41.601385 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" event={"ID":"9295f36e-f2a8-4e63-95b6-1fd6fe05c731","Type":"ContainerStarted","Data":"ccbc36cc74dd032e082251f8edf37a7f93062be6f8ecd22c7fac5fe65069c6ec"} Jul 06 23:04:41.601827 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:41.601449 2572 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" event={"ID":"9295f36e-f2a8-4e63-95b6-1fd6fe05c731","Type":"ContainerStarted","Data":"acc3313fa02a2653a5db7df0b31ed6ea709d70e3ba78f1b7a222bc9ef4d45f6f"} Jul 06 23:04:41.601827 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:41.601536 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:41.621725 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:41.621652 2572 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" podStartSLOduration=1.6216261360000002 podStartE2EDuration="1.621626136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 23:04:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 23:04:41.619712866 +0000 UTC m=+2413.923881530" watchObservedRunningTime="2026-07-06 23:04:41.621626136 +0000 UTC m=+2413.925794797" Jul 06 23:04:42.427213 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:42.427188 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2hg7t_b3aea0eb-2b2f-48f0-b7c0-e4b899544931/dns/0.log" Jul 06 23:04:42.448743 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:42.448708 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2hg7t_b3aea0eb-2b2f-48f0-b7c0-e4b899544931/kube-rbac-proxy/0.log" Jul 06 23:04:42.588502 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:42.588472 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n88h9_94e520a8-48a2-4f5e-9dce-f6487ff55798/dns-node-resolver/0.log" Jul 06 23:04:43.193190 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:43.193161 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-ddk2j_d4338e08-6387-4995-8195-d0ab26a18e5b/node-ca/0.log" Jul 06 23:04:44.531576 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:44.531546 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6b7b4fbb-m8lxl_a00628a2-128d-4499-b556-f8ca7f43572b/kube-auth-proxy/0.log" Jul 06 23:04:44.646694 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:44.646658 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-km4lp_434b4986-8680-4a50-b734-dad514382810/payload-pre-processing/0.log" Jul 06 23:04:44.707929 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:44.707882 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-88j4z_092bfe7a-d748-44dc-80d2-31d5d215d726/payload-processing/0.log" Jul 06 23:04:45.455763 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:45.455733 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbqm9_e6125683-3f5d-4cc6-ad3d-84b96bae4530/serve-healthcheck-canary/0.log" Jul 06 23:04:46.171953 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:46.171919 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j4mg6_7fc4d630-709f-4f3e-a3ff-4b93fde2bb80/kube-rbac-proxy/0.log" Jul 06 23:04:46.192848 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:46.192821 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j4mg6_7fc4d630-709f-4f3e-a3ff-4b93fde2bb80/exporter/0.log" Jul 06 23:04:46.215475 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:46.215443 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-j4mg6_7fc4d630-709f-4f3e-a3ff-4b93fde2bb80/extractor/0.log" Jul 06 23:04:47.619696 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:47.619662 2572 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-768zc/perf-node-gather-daemonset-bv22z" Jul 06 23:04:48.709402 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:48.709309 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-h6l2m_c9548ebb-ca06-4375-be3e-bb36a9add5de/manager/0.log" Jul 06 23:04:48.752767 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:48.752364 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-69b48b77f8-dvmfq_d46a7afc-865a-4459-9ae3-aaf250fc623d/maas-api/0.log" Jul 06 23:04:48.775909 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:48.775879 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722965-qzgzs_217511b3-53ed-4fe0-a455-e931e28582c2/cleanup/0.log" Jul 06 23:04:48.802796 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:48.802764 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722980-cbcnx_f8f50348-004a-4d7d-970c-dadf4c647190/cleanup/0.log" Jul 06 23:04:48.938172 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:48.938135 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d57c96767-mhkrw_49cfd9ff-47de-4085-ab62-ec75ed383487/manager/0.log" Jul 06 23:04:49.069020 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:49.068931 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84b8787b6c-gjb52_e63e4f76-619a-4a3c-9292-e8e02ec385fa/manager/0.log" Jul 06 23:04:49.090088 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:49.090047 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84b8787b6c-gjb52_e63e4f76-619a-4a3c-9292-e8e02ec385fa/copy-manifests/0.log" Jul 06 23:04:49.302938 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:49.302906 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-bz7wl_94ca973b-3a14-47ba-b1ee-c5ab3dfadb06/postgres/0.log" Jul 06 23:04:50.819359 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:50.819334 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69b75b5694-shk4q_27a05754-4a61-4622-889d-6d2f68449e84/manager/0.log" Jul 06 23:04:50.844477 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:50.844382 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nhs2w_ca086f1b-7b41-45dc-b153-1c61465c02b3/openshift-lws-operator/0.log" Jul 06 23:04:51.136364 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:51.136291 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6b7b4fbb-m8lxl_a00628a2-128d-4499-b556-f8ca7f43572b/kube-auth-proxy/0.log" Jul 06 23:04:51.217421 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:51.217390 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-km4lp_434b4986-8680-4a50-b734-dad514382810/payload-pre-processing/0.log" Jul 06 23:04:51.266646 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:51.266578 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-88j4z_092bfe7a-d748-44dc-80d2-31d5d215d726/payload-processing/0.log" Jul 06 23:04:52.758506 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:52.758471 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c6b7b4fbb-m8lxl_a00628a2-128d-4499-b556-f8ca7f43572b/kube-auth-proxy/0.log" Jul 06 23:04:52.841684 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:52.841658 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-km4lp_434b4986-8680-4a50-b734-dad514382810/payload-pre-processing/0.log" Jul 06 23:04:52.892122 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:52.892086 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-88j4z_092bfe7a-d748-44dc-80d2-31d5d215d726/payload-processing/0.log" Jul 06 23:04:57.554891 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.554861 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/kube-multus-additional-cni-plugins/0.log" Jul 06 23:04:57.576356 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.576274 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/egress-router-binary-copy/0.log" Jul 06 23:04:57.605956 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.605929 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/cni-plugins/0.log" Jul 06 23:04:57.625625 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.625603 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/bond-cni-plugin/0.log" Jul 06 23:04:57.645652 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.645627 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/routeoverride-cni/0.log" Jul 06 23:04:57.665373 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.665349 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/whereabouts-cni-bincopy/0.log" Jul 06 23:04:57.685013 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.684990 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-lkm55_53b05de2-d740-42a4-886a-f10ec45dbef5/whereabouts-cni/0.log" Jul 06 23:04:57.927751 ip-10-0-132-23 kubenswrapper[2572]: I0706 23:04:57.927720 2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-km9xf_fc9d82d3-31c2-48be-9913-7bf5684d1e1f/kube-multus/0.log"