Jul 22 04:24:35.790445 ip-10-0-142-182 systemd[1]: Starting Kubernetes Kubelet... Jul 22 04:24:36.703295 ip-10-0-142-182 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 04:24:36.703295 ip-10-0-142-182 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 04:24:36.703295 ip-10-0-142-182 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 04:24:36.703295 ip-10-0-142-182 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 04:24:36.703295 ip-10-0-142-182 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 04:24:36.703940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.703355 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 04:24:36.709693 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709674 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:24:36.709693 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709690 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:24:36.709693 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709695 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709698 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709702 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709705 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709708 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709711 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709714 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709719 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709723 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709726 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709730 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709733 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709736 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709739 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709742 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709745 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709748 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709750 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709753 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709756 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:24:36.709776 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709759 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709762 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709765 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709767 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709770 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709773 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709776 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709778 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709781 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709793 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709796 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709801 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709803 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709806 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709809 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709812 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709814 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709817 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709819 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:24:36.710297 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709822 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709829 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709832 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709835 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709838 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709841 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709843 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709846 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709848 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709851 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709854 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709856 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709859 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709861 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709864 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709867 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709870 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709873 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709876 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709879 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:24:36.710810 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709881 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709884 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709887 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709890 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709892 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709895 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709899 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709903 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709906 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709908 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709911 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709914 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709916 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709919 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709922 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709925 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709927 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709930 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709934 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:24:36.711320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709937 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709941 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709944 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709947 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709950 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709953 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709956 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709958 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709962 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709965 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709968 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709971 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709973 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709976 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709979 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709981 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709984 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709987 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.709990 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:24:36.711804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710761 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710769 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710774 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710779 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710782 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710785 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710788 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710793 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710797 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710801 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710804 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710807 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710810 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710813 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710817 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710820 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710823 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710826 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710828 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:24:36.712334 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710831 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710834 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710838 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710842 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710844 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710847 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710850 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710853 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710856 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710859 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710861 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710864 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710867 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710870 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710873 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710875 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710880 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710884 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710887 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710890 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:24:36.712830 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710892 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710895 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710898 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710902 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710905 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710907 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710910 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710913 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710915 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710918 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710921 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710924 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710926 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710929 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710932 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710935 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710938 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710941 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710943 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710946 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:24:36.713392 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710949 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710952 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710954 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710957 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710959 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710962 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710965 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710967 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710970 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710973 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710975 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710978 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710981 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710983 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710986 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710990 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710993 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710995 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.710998 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711001 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:24:36.713902 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711004 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711007 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711009 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711012 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711014 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711017 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711021 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711024 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711026 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711029 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711032 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711034 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711037 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711040 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711043 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711045 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711048 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711051 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711054 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711057 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:24:36.714433 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711468 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711475 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711479 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711483 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711487 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711490 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711493 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711496 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711500 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711503 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711506 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711509 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711512 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711516 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711518 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711521 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711524 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711527 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711530 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:24:36.714929 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711533 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711536 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711539 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711543 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711545 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711548 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711550 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711553 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711556 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711558 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711561 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711563 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711566 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711569 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711572 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711574 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711577 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711580 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711582 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711585 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:24:36.715451 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711588 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711590 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711594 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711598 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711601 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711604 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711607 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711609 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711612 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711615 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711618 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711621 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711624 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711626 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711629 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711632 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711634 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711638 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711640 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711643 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:24:36.715967 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711646 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711649 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711651 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711654 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711657 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711659 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711662 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711665 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711667 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711670 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711672 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711675 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711678 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711680 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711683 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711686 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711689 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711692 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711695 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711697 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:24:36.716488 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711700 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711703 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711708 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711711 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711713 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711717 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711721 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711726 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711729 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711731 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711734 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711737 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711740 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711742 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711745 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711748 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711751 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711753 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711756 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.711761 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:24:36.716988 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711846 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711862 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711871 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711884 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711890 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711898 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711905 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711911 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711918 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711922 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711927 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711930 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711934 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711937 2566 flags.go:64] FLAG: --cgroup-root="" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711941 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711945 2566 flags.go:64] FLAG: --client-ca-file="" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711948 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711951 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711956 2566 flags.go:64] FLAG: --cluster-domain="" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711959 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711962 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711965 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711969 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711973 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 04:24:36.717527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711976 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711980 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711983 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711986 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711989 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711993 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.711996 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712001 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712005 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712008 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712012 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712015 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712019 2566 flags.go:64] FLAG: --enable-server="true" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712022 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712026 2566 flags.go:64] FLAG: --event-burst="100" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712029 2566 flags.go:64] FLAG: --event-qps="50" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712032 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712036 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712039 2566 flags.go:64] FLAG: --eviction-hard="" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712043 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712047 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712050 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712055 2566 flags.go:64] FLAG: --eviction-soft="" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712059 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712062 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 04:24:36.718250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712065 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712069 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712072 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712075 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712078 2566 flags.go:64] FLAG: --feature-gates="" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712098 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712104 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712107 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712111 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712114 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712118 2566 flags.go:64] FLAG: --help="false" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712121 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-142-182.ec2.internal" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712124 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712127 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712132 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712136 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712140 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712157 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712161 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712164 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712167 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712170 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712173 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712177 2566 flags.go:64] FLAG: --kube-reserved="" Jul 22 04:24:36.718922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712180 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712183 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712187 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712190 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712193 2566 flags.go:64] FLAG: --lock-file="" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712198 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712201 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712205 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712211 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712214 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712217 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712220 2566 flags.go:64] FLAG: --logging-format="text" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712223 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712227 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712230 2566 flags.go:64] FLAG: --manifest-url="" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712233 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712237 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712241 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712245 2566 flags.go:64] FLAG: --max-pods="110" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712248 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712252 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712255 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712259 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712263 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712266 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 04:24:36.719573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712270 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712277 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712280 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712284 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712287 2566 flags.go:64] FLAG: --pod-cidr="" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712290 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712295 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712298 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712302 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712305 2566 flags.go:64] FLAG: --port="10250" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712309 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712312 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-051342a59834971e9" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712315 2566 flags.go:64] FLAG: --qos-reserved="" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712321 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712324 2566 flags.go:64] FLAG: --register-node="true" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712328 2566 flags.go:64] FLAG: --register-with-taints="" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712331 2566 flags.go:64] FLAG: --registry-burst="10" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712335 2566 flags.go:64] FLAG: --registry-qps="5" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712338 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712341 2566 flags.go:64] FLAG: --reserved-memory="" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712345 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712348 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712352 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712355 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712358 2566 flags.go:64] FLAG: --runonce="false" Jul 22 04:24:36.720276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712362 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712365 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712368 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712372 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712376 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712379 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712383 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712387 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712390 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712393 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712396 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712399 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712402 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712405 2566 flags.go:64] FLAG: --system-cgroups="" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712409 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712414 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712417 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712421 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712429 2566 flags.go:64] FLAG: --tls-min-version="" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712432 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712437 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712441 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712444 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712447 2566 flags.go:64] FLAG: --v="2" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712451 2566 flags.go:64] FLAG: --version="false" Jul 22 04:24:36.720925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712455 2566 flags.go:64] FLAG: --vmodule="" Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712459 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.712462 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712602 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712606 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712610 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712613 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712616 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712618 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712621 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712624 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712629 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712633 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712636 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712639 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712642 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712645 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712647 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712650 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:24:36.721601 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712653 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712656 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712658 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712661 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712664 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712667 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712674 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712677 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712682 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712684 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712687 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712690 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712693 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712695 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712698 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712701 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712703 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712706 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712709 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712711 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:24:36.722075 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712714 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712717 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712719 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712722 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712727 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712729 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712732 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712735 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712737 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712740 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712743 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712745 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712748 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712750 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712753 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712756 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712759 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712761 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712764 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712767 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:24:36.722682 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712772 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712774 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712777 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712780 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712783 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712786 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712788 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712791 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712794 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712796 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712799 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712802 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712804 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712807 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712809 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712812 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712816 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712819 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712822 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712824 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:24:36.723271 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712827 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712831 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712835 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712839 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712842 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712845 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712848 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712851 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712855 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712858 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712862 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712864 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712869 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712872 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712875 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712878 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712880 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712883 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712886 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:24:36.723785 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712888 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712891 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712894 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.712896 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.713575 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.721696 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.721715 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.721732 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.721739 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721809 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721815 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721819 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721822 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721825 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721828 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:24:36.724278 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721831 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721834 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721837 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721840 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721842 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721846 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721848 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721851 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721854 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721857 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721860 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721863 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721865 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721868 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721871 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721874 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721877 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721880 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721883 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721885 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:24:36.724664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721888 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721891 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721894 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721904 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721907 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721910 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721913 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721916 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721918 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721921 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721924 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721927 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721930 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721932 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721937 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721941 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721944 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721947 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721950 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721953 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:24:36.725191 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721957 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721959 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721962 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721965 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721968 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721971 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721974 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721976 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721979 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721982 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721984 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721987 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721989 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721992 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721995 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.721999 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722008 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722011 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722014 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722017 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:24:36.725731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722019 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722022 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722025 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722027 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722030 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722033 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722035 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722038 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722040 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722045 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722049 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722051 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722054 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722057 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722060 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722062 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722065 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722068 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722071 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:24:36.726326 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722073 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722076 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722079 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722082 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722086 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722089 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722093 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722096 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722100 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722103 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722116 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722119 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722122 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722125 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.722131 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 04:24:36.726799 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722299 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722304 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722307 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722310 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722313 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722316 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722319 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722322 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722324 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722327 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722329 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722332 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722335 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722337 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722340 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722343 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722345 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722348 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722351 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722353 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 04:24:36.727243 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722357 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722361 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722364 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722366 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722369 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722372 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722375 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722378 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722388 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722392 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722395 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722398 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722401 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722404 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722407 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722409 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722412 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722415 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722417 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 04:24:36.727753 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722420 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722422 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722425 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722428 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722430 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722433 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722436 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722438 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722441 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722444 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722447 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722449 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722452 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722455 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722457 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722460 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722463 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722465 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722468 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722471 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 04:24:36.728245 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722474 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722477 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722486 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722489 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722491 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722494 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722497 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722499 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722502 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722505 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722507 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722510 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722513 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722515 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722518 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722521 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722523 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722526 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722528 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722531 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 04:24:36.728751 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722534 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722537 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722539 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722542 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722544 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722549 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722552 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722555 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722559 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722562 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722565 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722568 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722570 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722573 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722576 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722584 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722587 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722590 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722592 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 04:24:36.729273 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:36.722595 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 04:24:36.729759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.722600 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 04:24:36.729759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.722730 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 04:24:36.729759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.725341 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 04:24:36.729759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.726775 2566 server.go:1019] "Starting client certificate rotation" Jul 22 04:24:36.729759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.726883 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 04:24:36.729759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.727446 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 04:24:36.730388 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.730373 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 04:24:36.735951 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.735925 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 04:24:36.746672 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.746656 2566 log.go:25] "Validated CRI v1 runtime API" Jul 22 04:24:36.748998 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.748984 2566 log.go:25] "Validated CRI v1 image API" Jul 22 04:24:36.750686 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.750669 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 04:24:36.753290 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.753266 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7b977a1b-b589-4020-9afb-6c1803fb8b55:/dev/nvme0n1p3 8eedad98-dd81-4f11-b718-6ced51bc253e:/dev/nvme0n1p4] Jul 22 04:24:36.753343 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.753290 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 04:24:36.758839 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.758736 2566 manager.go:216] Machine: {Timestamp:2026-07-22 04:24:36.757435893 +0000 UTC m=+0.846715058 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099867 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e57d395ca6ed76e5b65aff8b8e368 SystemUUID:ec2e57d3-95ca-6ed7-6e5b-65aff8b8e368 BootID:ba6c3f47-0cd2-4c45-b8a3-34470e188b29 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:bb:f7:42:2b:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:bb:f7:42:2b:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:32:85:6f:2d:5b:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 04:24:36.758839 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.758838 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 04:24:36.758961 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.758949 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 04:24:36.760286 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.760272 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 04:24:36.762265 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.762241 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 04:24:36.762403 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.762271 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-182.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 04:24:36.763083 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.763073 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 04:24:36.763116 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.763086 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 04:24:36.763116 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.763094 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 04:24:36.763850 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.763839 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 04:24:36.765908 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.765897 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 04:24:36.769571 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769559 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 22 04:24:36.769698 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769688 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 04:24:36.769745 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769737 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 22 04:24:36.769776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769750 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 04:24:36.769776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769761 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 04:24:36.769776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769770 2566 kubelet.go:398] "Adding apiserver pod source" Jul 22 04:24:36.769881 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.769779 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 04:24:36.770111 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.770099 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 04:24:36.770170 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.770118 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 04:24:36.772472 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.772458 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 04:24:36.774692 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.774677 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 04:24:36.774747 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.774710 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 04:24:36.776502 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776490 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 04:24:36.776547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776508 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 04:24:36.776547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776514 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 04:24:36.776547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776520 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 04:24:36.776547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776534 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 04:24:36.776547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776543 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 04:24:36.776688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776552 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 04:24:36.776688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776558 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 04:24:36.776688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776564 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 04:24:36.776688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776571 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 04:24:36.776688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776580 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 04:24:36.776688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.776586 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 04:24:36.778246 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.778234 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 04:24:36.778246 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.778247 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 04:24:36.782274 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.782006 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 04:24:36.782274 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.782062 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 04:24:36.782274 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.782063 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 04:24:36.782274 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.782116 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-182.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 04:24:36.782274 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.782141 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 04:24:36.782590 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.782569 2566 server.go:1268] "Started kubelet" Jul 22 04:24:36.783367 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.783345 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-182.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 04:24:36.783512 ip-10-0-142-182 systemd[1]: Started Kubernetes Kubelet. Jul 22 04:24:36.784904 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.784713 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 04:24:36.784958 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.784908 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 22 04:24:36.789758 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.789725 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 04:24:36.795815 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.795800 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 04:24:36.795902 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.795803 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 04:24:36.795960 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.795918 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:36.796695 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.796677 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 04:24:36.796781 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.796743 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 04:24:36.796781 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.796766 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 04:24:36.797018 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.796880 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 22 04:24:36.797018 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.796891 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 22 04:24:36.797249 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797218 2566 factory.go:153] Registering CRI-O factory Jul 22 04:24:36.797249 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797239 2566 factory.go:223] Registration of the crio container factory successfully Jul 22 04:24:36.797436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797289 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 04:24:36.797436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797299 2566 factory.go:55] Registering systemd factory Jul 22 04:24:36.797436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797307 2566 factory.go:223] Registration of the systemd container factory successfully Jul 22 04:24:36.797436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797330 2566 factory.go:103] Registering Raw factory Jul 22 04:24:36.797436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797346 2566 manager.go:1196] Started watching for new ooms in manager Jul 22 04:24:36.797776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.797759 2566 manager.go:349] Starting recovery of all containers Jul 22 04:24:36.799404 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.799372 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-182.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 04:24:36.799589 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.799570 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 04:24:36.800408 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.799373 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-182.ec2.internal.18c481fd30426a1d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-182.ec2.internal,UID:ip-10-0-142-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-182.ec2.internal,},FirstTimestamp:2026-07-22 04:24:36.781926941 +0000 UTC m=+0.871206114,LastTimestamp:2026-07-22 04:24:36.781926941 +0000 UTC m=+0.871206114,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-182.ec2.internal,}" Jul 22 04:24:36.807466 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.807448 2566 manager.go:354] Recovery completed Jul 22 04:24:36.807595 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.807573 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hz8rq" Jul 22 04:24:36.811801 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.811788 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:36.813166 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.813133 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hz8rq" Jul 22 04:24:36.813658 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.813644 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:36.813708 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.813673 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:36.813708 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.813684 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:36.814288 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814270 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 04:24:36.814288 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814282 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:36.814418 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814285 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 04:24:36.814418 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814361 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 22 04:24:36.814937 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814922 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:36.814987 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814948 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:36.814987 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.814958 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:36.815862 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.815845 2566 policy_none.go:49] "None policy: Start" Jul 22 04:24:36.815940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.815866 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 04:24:36.815940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.815881 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 04:24:36.816019 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.815869 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-182.ec2.internal.18c481fd32269ec8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-182.ec2.internal,UID:ip-10-0-142-182.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-182.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-182.ec2.internal,},FirstTimestamp:2026-07-22 04:24:36.813659848 +0000 UTC m=+0.902939017,LastTimestamp:2026-07-22 04:24:36.813659848 +0000 UTC m=+0.902939017,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-182.ec2.internal,}" Jul 22 04:24:36.816967 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.816955 2566 policy_none.go:47] "Start" Jul 22 04:24:36.862716 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.862701 2566 manager.go:337] "Starting Device Plugin manager" Jul 22 04:24:36.862810 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.862737 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 04:24:36.862810 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.862747 2566 server.go:87] "Starting device plugin registration server" Jul 22 04:24:36.862998 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.862985 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 04:24:36.863064 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.863001 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 04:24:36.863168 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.863139 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 04:24:36.863249 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.863233 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 04:24:36.863249 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.863248 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 04:24:36.864517 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.864496 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 04:24:36.864592 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.864544 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:36.923418 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.923393 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 04:24:36.924657 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.924632 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 04:24:36.924746 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.924674 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 04:24:36.924746 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.924695 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 04:24:36.924746 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.924727 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 04:24:36.926912 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.926894 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:24:36.963974 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.963955 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:36.964814 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.964793 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:36.964893 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.964823 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:36.964893 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.964835 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:36.964893 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.964857 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-182.ec2.internal" Jul 22 04:24:36.975209 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:36.975192 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-182.ec2.internal" Jul 22 04:24:36.975285 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.975212 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-182.ec2.internal\": node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:36.999055 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:36.999032 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.024823 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.024798 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal"] Jul 22 04:24:37.024905 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.024864 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:37.026071 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.026055 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:37.026187 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.026084 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:37.026187 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.026099 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:37.027632 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.027617 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:37.027746 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.027731 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.027796 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.027760 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:37.028370 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.028358 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:37.028438 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.028379 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:37.028438 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.028388 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:37.028438 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.028409 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:37.028438 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.028432 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:37.028583 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.028442 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:37.029707 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.029691 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.029750 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.029724 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 04:24:37.030802 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.030787 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientMemory" Jul 22 04:24:37.030874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.030813 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 04:24:37.030874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.030823 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeHasSufficientPID" Jul 22 04:24:37.053787 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.053738 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-182.ec2.internal\" not found" node="ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.058374 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.058358 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-182.ec2.internal\" not found" node="ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.099534 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.099505 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.198490 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.198449 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe37cdaa3f7b18022e2f8cd5afc0ca1e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal\" (UID: \"fe37cdaa3f7b18022e2f8cd5afc0ca1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.198490 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.198493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe37cdaa3f7b18022e2f8cd5afc0ca1e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal\" (UID: \"fe37cdaa3f7b18022e2f8cd5afc0ca1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.198692 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.198523 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c19c00cd70e4577ed80789f2afae047c-config\") pod \"kube-apiserver-proxy-ip-10-0-142-182.ec2.internal\" (UID: \"c19c00cd70e4577ed80789f2afae047c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.200716 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.200695 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.298725 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.298697 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe37cdaa3f7b18022e2f8cd5afc0ca1e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal\" (UID: \"fe37cdaa3f7b18022e2f8cd5afc0ca1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.298803 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.298714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fe37cdaa3f7b18022e2f8cd5afc0ca1e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal\" (UID: \"fe37cdaa3f7b18022e2f8cd5afc0ca1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.298803 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.298767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe37cdaa3f7b18022e2f8cd5afc0ca1e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal\" (UID: \"fe37cdaa3f7b18022e2f8cd5afc0ca1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.298803 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.298790 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c19c00cd70e4577ed80789f2afae047c-config\") pod \"kube-apiserver-proxy-ip-10-0-142-182.ec2.internal\" (UID: \"c19c00cd70e4577ed80789f2afae047c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.298901 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.298829 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c19c00cd70e4577ed80789f2afae047c-config\") pod \"kube-apiserver-proxy-ip-10-0-142-182.ec2.internal\" (UID: \"c19c00cd70e4577ed80789f2afae047c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.298901 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.298867 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe37cdaa3f7b18022e2f8cd5afc0ca1e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal\" (UID: \"fe37cdaa3f7b18022e2f8cd5afc0ca1e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.300876 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.300854 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.362825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.362768 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.362825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.362766 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.401517 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.401488 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.502824 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.502789 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.603428 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.603400 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.704531 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:37.704455 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-182.ec2.internal\" not found" Jul 22 04:24:37.726776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.726748 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 04:24:37.726945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.726925 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 04:24:37.795710 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.795686 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:24:37.796626 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.796607 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 04:24:37.796711 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.796632 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.807088 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.807068 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 04:24:37.810080 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.810063 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 04:24:37.812136 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.812120 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" Jul 22 04:24:37.818591 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.818568 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 04:19:37 +0000 UTC" deadline="2028-03-14 23:04:07.425360641 +0000 UTC" Jul 22 04:24:37.818591 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.818590 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14442h39m29.606773147s" Jul 22 04:24:37.821759 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.821743 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 04:24:37.829719 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.829702 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8mhrx" Jul 22 04:24:37.835972 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.835950 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8mhrx" Jul 22 04:24:37.945387 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:37.945248 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:24:38.083676 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:38.083644 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19c00cd70e4577ed80789f2afae047c.slice/crio-241268bac337e31aa4ff8e19bec4e442edca1abf1e3f10ec40aeaf67ef522681 WatchSource:0}: Error finding container 241268bac337e31aa4ff8e19bec4e442edca1abf1e3f10ec40aeaf67ef522681: Status 404 returned error can't find the container with id 241268bac337e31aa4ff8e19bec4e442edca1abf1e3f10ec40aeaf67ef522681 Jul 22 04:24:38.087120 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.087101 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:24:38.094527 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:38.094504 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe37cdaa3f7b18022e2f8cd5afc0ca1e.slice/crio-9fee535fca8cfba59548b1d5f970bc2bc3fde72c3782783c6190c278b1c031d5 WatchSource:0}: Error finding container 9fee535fca8cfba59548b1d5f970bc2bc3fde72c3782783c6190c278b1c031d5: Status 404 returned error can't find the container with id 9fee535fca8cfba59548b1d5f970bc2bc3fde72c3782783c6190c278b1c031d5 Jul 22 04:24:38.139605 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.139579 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:24:38.632929 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.632897 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 04:24:38.770948 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.770912 2566 apiserver.go:52] "Watching apiserver" Jul 22 04:24:38.779282 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.778093 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 04:24:38.779282 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.778658 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk","openshift-cluster-node-tuning-operator/tuned-m7ljf","openshift-multus/network-metrics-daemon-4fnp2","openshift-network-operator/iptables-alerter-m29kf","kube-system/global-pull-secret-syncer-jvssr","openshift-dns/node-resolver-96kj6","openshift-image-registry/node-ca-7vpdn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal","openshift-multus/multus-additional-cni-plugins-crnp4","openshift-multus/multus-z9c8z","openshift-network-diagnostics/network-check-target-wlhqp","openshift-ovn-kubernetes/ovnkube-node-qnb42","kube-system/konnectivity-agent-s6bkv"] Jul 22 04:24:38.781663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.781635 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.784018 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.783997 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 04:24:38.784184 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.784094 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.784403 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.784383 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.784721 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.784699 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.784820 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.784761 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jjx4g\"" Jul 22 04:24:38.786228 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.786208 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.786693 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.786675 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.786798 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.786782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:38.786908 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.786891 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gfz2j\"" Jul 22 04:24:38.786969 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.786889 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:38.789998 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.789974 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.791824 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.791805 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.791906 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.791874 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:38.794877 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.794382 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l4qb5\"" Jul 22 04:24:38.794877 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.794418 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.794877 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.794525 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 04:24:38.794877 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.794385 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.794877 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.794795 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.796431 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.796410 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.797056 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.797036 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ds8pj\"" Jul 22 04:24:38.797163 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.797041 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.797227 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.797043 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.798478 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.798445 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.798868 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.798848 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 04:24:38.798990 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.798972 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-659m8\"" Jul 22 04:24:38.799258 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.799240 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.799606 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.799589 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.803223 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.803200 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 04:24:38.806021 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.805990 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 04:24:38.806134 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.806062 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.806134 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.806068 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4xh8h\"" Jul 22 04:24:38.806258 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.806000 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.806326 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.806310 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 04:24:38.807509 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.807449 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24dwn\" (UniqueName: \"kubernetes.io/projected/cc16b524-beac-46e2-a269-a082228fe97c-kube-api-access-24dwn\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-system-cni-dir\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809228 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cnibin\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-etc-selinux\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-lib-modules\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a864c73c-bf76-41a9-a509-85976d29df5e-dbus\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg956\" (UniqueName: \"kubernetes.io/projected/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-kube-api-access-xg956\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-os-release\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wcwj\" (UniqueName: \"kubernetes.io/projected/182c8bce-0ce8-4dc5-8a93-42cd57848c35-kube-api-access-4wcwj\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cb9c999-df21-4553-b788-e757369a9963-iptables-alerter-script\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cb9c999-df21-4553-b788-e757369a9963-host-slash\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wrnc\" (UniqueName: \"kubernetes.io/projected/c091b74a-8aef-4cb2-8042-67b4e153cdb7-kube-api-access-2wrnc\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809631 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-socket-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809654 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cc16b524-beac-46e2-a269-a082228fe97c-etc-tuned\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.810192 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809704 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-tmp-dir\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809730 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysctl-d\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809754 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysctl-conf\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-sys\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc16b524-beac-46e2-a269-a082228fe97c-tmp\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809824 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vk842\" (UniqueName: \"kubernetes.io/projected/256b8fe7-db10-4e43-9d5c-d75acc517fb2-kube-api-access-vk842\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809851 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-hosts-file\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-registration-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809908 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-sys-fs\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-systemd\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809956 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-run\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.809981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-var-lib-kubelet\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810005 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb43c786-c2c2-459e-bb69-7a661f7216ae-serviceca\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-modprobe-d\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-kubernetes\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810079 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-host\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln26z\" (UniqueName: \"kubernetes.io/projected/bb43c786-c2c2-459e-bb69-7a661f7216ae-kube-api-access-ln26z\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810157 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-497qm\" (UniqueName: \"kubernetes.io/projected/0cb9c999-df21-4553-b788-e757369a9963-kube-api-access-497qm\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810186 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a864c73c-bf76-41a9-a509-85976d29df5e-kubelet-config\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810213 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysconfig\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810320 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb43c786-c2c2-459e-bb69-7a661f7216ae-host\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810346 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810400 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-device-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810892 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.810970 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-z8td5\"" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.811061 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:38.811918 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.811150 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:38.812632 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.812274 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.814313 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.814292 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 04:24:38.814977 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.814953 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:38.815078 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.814998 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 04:24:38.815078 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.815026 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2cdrx\"" Jul 22 04:24:38.815838 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.815814 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 04:24:38.816094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.816075 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 04:24:38.816573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.816322 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 04:24:38.816573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.816537 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 04:24:38.818952 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.817487 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 04:24:38.818952 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.817890 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9l5d2\"" Jul 22 04:24:38.818952 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.818152 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 04:24:38.838322 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.838238 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 04:19:37 +0000 UTC" deadline="2028-04-19 19:05:14.83887808 +0000 UTC" Jul 22 04:24:38.838322 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.838282 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15302h40m36.000600009s" Jul 22 04:24:38.898030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.897932 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 04:24:38.910641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.910757 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb43c786-c2c2-459e-bb69-7a661f7216ae-host\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.910757 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910694 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-system-cni-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.910757 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.910757 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-cni-binary-copy\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.910757 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bb43c786-c2c2-459e-bb69-7a661f7216ae-host\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910751 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-cni-multus\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910796 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-ovnkube-config\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910825 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910853 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24dwn\" (UniqueName: \"kubernetes.io/projected/cc16b524-beac-46e2-a269-a082228fe97c-kube-api-access-24dwn\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910880 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-ovn\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910909 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910935 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a864c73c-bf76-41a9-a509-85976d29df5e-dbus\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.910973 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cts84\" (UniqueName: \"kubernetes.io/projected/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-kube-api-access-cts84\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-os-release\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911028 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wcwj\" (UniqueName: \"kubernetes.io/projected/182c8bce-0ce8-4dc5-8a93-42cd57848c35-kube-api-access-4wcwj\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cb9c999-df21-4553-b788-e757369a9963-iptables-alerter-script\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.911094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911080 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-kubelet\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.911879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-ovnkube-script-lib\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.911879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911156 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:38.911879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911184 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-cnibin\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.912688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.912661 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.912859 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.912832 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:38.912972 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.912957 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:39.41291282 +0000 UTC m=+3.502191982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:38.913630 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.913597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.913825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.913807 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/a864c73c-bf76-41a9-a509-85976d29df5e-dbus\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.914246 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-slash\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.914357 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-etc-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.914357 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.914357 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-tmp-dir\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.914531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc16b524-beac-46e2-a269-a082228fe97c-tmp\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.914531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914397 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/0cb9c999-df21-4553-b788-e757369a9963-iptables-alerter-script\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.914531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914415 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vk842\" (UniqueName: \"kubernetes.io/projected/256b8fe7-db10-4e43-9d5c-d75acc517fb2-kube-api-access-vk842\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:38.914531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914444 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70691c50-ec06-4b34-966d-0730f0ebca13-konnectivity-ca\") pod \"konnectivity-agent-s6bkv\" (UID: \"70691c50-ec06-4b34-966d-0730f0ebca13\") " pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:38.914531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.911211 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-os-release\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.914531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914516 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-conf-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.914848 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914793 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-tmp-dir\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.914903 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914850 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-multus-certs\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.914903 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914848 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 04:24:38.914903 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914887 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:38.915050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-socket-dir-parent\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.915050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.914972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-registration-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.915050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-systemd\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.915050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915032 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-run\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-modprobe-d\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915182 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-run\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-kubernetes\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915251 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln26z\" (UniqueName: \"kubernetes.io/projected/bb43c786-c2c2-459e-bb69-7a661f7216ae-kube-api-access-ln26z\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915285 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-netns\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-var-lib-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.915372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915348 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-cni-bin\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.915773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915396 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.915773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915423 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysconfig\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.915773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.915773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-device-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.915773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.915773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915770 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-etc-kubernetes\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-kubelet\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-log-socket\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915869 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-system-cni-dir\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915901 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cnibin\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-etc-selinux\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-lib-modules\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.915992 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-daemon-config\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-systemd\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916060 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg956\" (UniqueName: \"kubernetes.io/projected/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-kube-api-access-xg956\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916093 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cb9c999-df21-4553-b788-e757369a9963-host-slash\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.916167 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916174 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-node-log\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916205 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz9bh\" (UniqueName: \"kubernetes.io/projected/e8971da1-28ee-4d18-bc40-8224326e12fc-kube-api-access-qz9bh\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wrnc\" (UniqueName: \"kubernetes.io/projected/c091b74a-8aef-4cb2-8042-67b4e153cdb7-kube-api-access-2wrnc\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.916709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916291 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-socket-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.916709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cc16b524-beac-46e2-a269-a082228fe97c-etc-tuned\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-systemd-units\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916774 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-run-netns\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916835 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8971da1-28ee-4d18-bc40-8224326e12fc-ovn-node-metrics-cert\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysctl-d\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916901 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysctl-conf\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.916950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916931 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-sys\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70691c50-ec06-4b34-966d-0730f0ebca13-agent-certs\") pod \"konnectivity-agent-s6bkv\" (UID: \"70691c50-ec06-4b34-966d-0730f0ebca13\") " pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.916989 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-cni-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-cni-netd\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917049 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-k8s-cni-cncf-io\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917082 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-hosts-file\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917107 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-sys-fs\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-var-lib-kubelet\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917201 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb43c786-c2c2-459e-bb69-7a661f7216ae-serviceca\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-497qm\" (UniqueName: \"kubernetes.io/projected/0cb9c999-df21-4553-b788-e757369a9963-kube-api-access-497qm\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a864c73c-bf76-41a9-a509-85976d29df5e-kubelet-config\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.917302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-host\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-os-release\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917383 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-cni-bin\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917411 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-hostroot\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917445 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-env-overrides\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917622 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-socket-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.917771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917682 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-device-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.918189 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917809 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-modprobe-d\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.918189 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917878 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-registration-dir\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.918189 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917925 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-systemd\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.918189 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.917971 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-kubernetes\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.918719 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.918667 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cni-binary-copy\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.918819 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.918769 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysconfig\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.918885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.918823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-system-cni-dir\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.918943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.918904 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c091b74a-8aef-4cb2-8042-67b4e153cdb7-cnibin\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:38.919002 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.918991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-etc-selinux\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.919104 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919058 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-lib-modules\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.919301 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0cb9c999-df21-4553-b788-e757369a9963-host-slash\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.919514 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919493 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysctl-conf\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.919575 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919561 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-var-lib-kubelet\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.919627 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919582 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-sys\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.919700 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919681 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-etc-sysctl-d\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.919751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919716 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/182c8bce-0ce8-4dc5-8a93-42cd57848c35-sys-fs\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.919751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919686 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/a864c73c-bf76-41a9-a509-85976d29df5e-kubelet-config\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:38.919864 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.919837 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:38.919995 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.919887 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc16b524-beac-46e2-a269-a082228fe97c-host\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.919995 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:38.919935 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:24:39.41991563 +0000 UTC m=+3.509194795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:38.920276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.920252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-hosts-file\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.920371 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.920352 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/bb43c786-c2c2-459e-bb69-7a661f7216ae-serviceca\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.921744 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.921629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24dwn\" (UniqueName: \"kubernetes.io/projected/cc16b524-beac-46e2-a269-a082228fe97c-kube-api-access-24dwn\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.922969 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.922920 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/cc16b524-beac-46e2-a269-a082228fe97c-etc-tuned\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.923372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.923330 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cc16b524-beac-46e2-a269-a082228fe97c-tmp\") pod \"tuned-m7ljf\" (UID: \"cc16b524-beac-46e2-a269-a082228fe97c\") " pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:38.930805 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.930780 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg956\" (UniqueName: \"kubernetes.io/projected/3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3-kube-api-access-xg956\") pod \"node-resolver-96kj6\" (UID: \"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3\") " pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:38.931568 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.931543 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln26z\" (UniqueName: \"kubernetes.io/projected/bb43c786-c2c2-459e-bb69-7a661f7216ae-kube-api-access-ln26z\") pod \"node-ca-7vpdn\" (UID: \"bb43c786-c2c2-459e-bb69-7a661f7216ae\") " pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:38.933161 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.933094 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" event={"ID":"fe37cdaa3f7b18022e2f8cd5afc0ca1e","Type":"ContainerStarted","Data":"9fee535fca8cfba59548b1d5f970bc2bc3fde72c3782783c6190c278b1c031d5"} Jul 22 04:24:38.933738 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.933649 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vk842\" (UniqueName: \"kubernetes.io/projected/256b8fe7-db10-4e43-9d5c-d75acc517fb2-kube-api-access-vk842\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:38.934347 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.934322 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wcwj\" (UniqueName: \"kubernetes.io/projected/182c8bce-0ce8-4dc5-8a93-42cd57848c35-kube-api-access-4wcwj\") pod \"aws-ebs-csi-driver-node-l5fmk\" (UID: \"182c8bce-0ce8-4dc5-8a93-42cd57848c35\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:38.935120 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.935095 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-497qm\" (UniqueName: \"kubernetes.io/projected/0cb9c999-df21-4553-b788-e757369a9963-kube-api-access-497qm\") pod \"iptables-alerter-m29kf\" (UID: \"0cb9c999-df21-4553-b788-e757369a9963\") " pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:38.935543 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.935518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" event={"ID":"c19c00cd70e4577ed80789f2afae047c","Type":"ContainerStarted","Data":"241268bac337e31aa4ff8e19bec4e442edca1abf1e3f10ec40aeaf67ef522681"} Jul 22 04:24:38.936191 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:38.936168 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wrnc\" (UniqueName: \"kubernetes.io/projected/c091b74a-8aef-4cb2-8042-67b4e153cdb7-kube-api-access-2wrnc\") pod \"multus-additional-cni-plugins-crnp4\" (UID: \"c091b74a-8aef-4cb2-8042-67b4e153cdb7\") " pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:39.018717 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018674 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-node-log\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018725 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz9bh\" (UniqueName: \"kubernetes.io/projected/e8971da1-28ee-4d18-bc40-8224326e12fc-kube-api-access-qz9bh\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018753 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-node-log\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018769 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-systemd-units\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-run-netns\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018822 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8971da1-28ee-4d18-bc40-8224326e12fc-ovn-node-metrics-cert\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.018913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70691c50-ec06-4b34-966d-0730f0ebca13-agent-certs\") pod \"konnectivity-agent-s6bkv\" (UID: \"70691c50-ec06-4b34-966d-0730f0ebca13\") " pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-cni-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018966 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-run-netns\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.018984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-cni-netd\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019026 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-k8s-cni-cncf-io\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-os-release\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019079 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-systemd-units\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019103 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-cni-bin\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019091 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-cni-netd\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-hostroot\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-env-overrides\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-system-cni-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019409 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-cni-binary-copy\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-cni-multus\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-ovnkube-config\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-ovn\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019530 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cts84\" (UniqueName: \"kubernetes.io/projected/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-kube-api-access-cts84\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-kubelet\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019585 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-ovnkube-script-lib\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-os-release\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-cnibin\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-k8s-cni-cncf-io\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019682 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-slash\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019703 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-cnibin\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019715 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-etc-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019744 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019751 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-cni-multus\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.019784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019213 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019777 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70691c50-ec06-4b34-966d-0730f0ebca13-konnectivity-ca\") pod \"konnectivity-agent-s6bkv\" (UID: \"70691c50-ec06-4b34-966d-0730f0ebca13\") " pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019806 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-conf-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-multus-certs\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019890 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-socket-dir-parent\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019973 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-slash\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019993 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-socket-dir-parent\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-netns\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-etc-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-var-lib-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-run-ovn-kubernetes\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-cni-bin\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-etc-kubernetes\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020158 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-kubelet\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020167 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-conf-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-log-socket\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020210 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-multus-certs\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.020680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020218 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-daemon-config\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-systemd\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020350 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019243 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-cni-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020448 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-cni-bin\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-hostroot\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020508 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-ovn\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020523 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-system-cni-dir\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.019192 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-cni-bin\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020754 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-var-lib-kubelet\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.020761 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-env-overrides\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021119 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-cni-binary-copy\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021200 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-host-run-netns\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021222 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-ovnkube-config\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021248 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-var-lib-openvswitch\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-log-socket\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-etc-kubernetes\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.021559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021342 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-run-systemd\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.022541 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021340 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e8971da1-28ee-4d18-bc40-8224326e12fc-host-kubelet\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.022541 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-multus-daemon-config\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.022541 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.021863 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/70691c50-ec06-4b34-966d-0730f0ebca13-konnectivity-ca\") pod \"konnectivity-agent-s6bkv\" (UID: \"70691c50-ec06-4b34-966d-0730f0ebca13\") " pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:39.022541 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.022122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e8971da1-28ee-4d18-bc40-8224326e12fc-ovnkube-script-lib\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.025213 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.025188 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/70691c50-ec06-4b34-966d-0730f0ebca13-agent-certs\") pod \"konnectivity-agent-s6bkv\" (UID: \"70691c50-ec06-4b34-966d-0730f0ebca13\") " pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:39.025641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.025552 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8971da1-28ee-4d18-bc40-8224326e12fc-ovn-node-metrics-cert\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.027843 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.027751 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:24:39.027843 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.027801 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:24:39.027843 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.027832 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:39.028129 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.027928 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:39.527888852 +0000 UTC m=+3.617168006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:39.032180 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.032127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz9bh\" (UniqueName: \"kubernetes.io/projected/e8971da1-28ee-4d18-bc40-8224326e12fc-kube-api-access-qz9bh\") pod \"ovnkube-node-qnb42\" (UID: \"e8971da1-28ee-4d18-bc40-8224326e12fc\") " pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.032293 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.032264 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cts84\" (UniqueName: \"kubernetes.io/projected/eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef-kube-api-access-cts84\") pod \"multus-z9c8z\" (UID: \"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef\") " pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.104143 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.104106 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" Jul 22 04:24:39.114471 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.114446 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" Jul 22 04:24:39.115416 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.115386 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod182c8bce_0ce8_4dc5_8a93_42cd57848c35.slice/crio-debf301303735708a3d60fba38af8094f493e33e800cc4f8bcc8a6c8b3834f34 WatchSource:0}: Error finding container debf301303735708a3d60fba38af8094f493e33e800cc4f8bcc8a6c8b3834f34: Status 404 returned error can't find the container with id debf301303735708a3d60fba38af8094f493e33e800cc4f8bcc8a6c8b3834f34 Jul 22 04:24:39.125518 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.125348 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m29kf" Jul 22 04:24:39.126039 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.126010 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc16b524_beac_46e2_a269_a082228fe97c.slice/crio-4e5405b2d81d2ba9a87d049c31f97bfd5054c252e5e7f8f21ed6ab3ccfd65038 WatchSource:0}: Error finding container 4e5405b2d81d2ba9a87d049c31f97bfd5054c252e5e7f8f21ed6ab3ccfd65038: Status 404 returned error can't find the container with id 4e5405b2d81d2ba9a87d049c31f97bfd5054c252e5e7f8f21ed6ab3ccfd65038 Jul 22 04:24:39.132312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.132147 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-96kj6" Jul 22 04:24:39.135877 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.135848 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb9c999_df21_4553_b788_e757369a9963.slice/crio-9532f830cf5a6a44a77be54dd337e0fa87e9ac1952856ce1548322dd94c240ec WatchSource:0}: Error finding container 9532f830cf5a6a44a77be54dd337e0fa87e9ac1952856ce1548322dd94c240ec: Status 404 returned error can't find the container with id 9532f830cf5a6a44a77be54dd337e0fa87e9ac1952856ce1548322dd94c240ec Jul 22 04:24:39.140040 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.140018 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7vpdn" Jul 22 04:24:39.143272 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.143247 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a848e11_271b_4c2f_bdb3_4d4ac66fc3c3.slice/crio-fc87d5edf031e0bb5172319d4c1ed599f313a5773a10bdaa0864ce22b74cc10b WatchSource:0}: Error finding container fc87d5edf031e0bb5172319d4c1ed599f313a5773a10bdaa0864ce22b74cc10b: Status 404 returned error can't find the container with id fc87d5edf031e0bb5172319d4c1ed599f313a5773a10bdaa0864ce22b74cc10b Jul 22 04:24:39.149927 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.149898 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-crnp4" Jul 22 04:24:39.152320 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.152291 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb43c786_c2c2_459e_bb69_7a661f7216ae.slice/crio-5e96d1ba401ed10c491b7aeb3831a2aa66e2b992c31800c5fe8d3f6619ee6310 WatchSource:0}: Error finding container 5e96d1ba401ed10c491b7aeb3831a2aa66e2b992c31800c5fe8d3f6619ee6310: Status 404 returned error can't find the container with id 5e96d1ba401ed10c491b7aeb3831a2aa66e2b992c31800c5fe8d3f6619ee6310 Jul 22 04:24:39.159259 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.159237 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z9c8z" Jul 22 04:24:39.164937 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.164907 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc091b74a_8aef_4cb2_8042_67b4e153cdb7.slice/crio-958a515b9e9fd8e518dcfff315698e8a8ad92f226d8e833908c279b6bf5b736c WatchSource:0}: Error finding container 958a515b9e9fd8e518dcfff315698e8a8ad92f226d8e833908c279b6bf5b736c: Status 404 returned error can't find the container with id 958a515b9e9fd8e518dcfff315698e8a8ad92f226d8e833908c279b6bf5b736c Jul 22 04:24:39.167679 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.167460 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:24:39.170663 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.170645 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeed6b91_c5c1_4555_b6a4_9d2bd34fe7ef.slice/crio-6631e6a159978920ac07f9692024857fb521be3a61c5fbe7724ea3722af58494 WatchSource:0}: Error finding container 6631e6a159978920ac07f9692024857fb521be3a61c5fbe7724ea3722af58494: Status 404 returned error can't find the container with id 6631e6a159978920ac07f9692024857fb521be3a61c5fbe7724ea3722af58494 Jul 22 04:24:39.174780 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.174763 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:24:39.178014 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.177990 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8971da1_28ee_4d18_bc40_8224326e12fc.slice/crio-204dbdad243dcc33834d1994705c494ec01bb8d4cea9042cb81e64442b8fd07c WatchSource:0}: Error finding container 204dbdad243dcc33834d1994705c494ec01bb8d4cea9042cb81e64442b8fd07c: Status 404 returned error can't find the container with id 204dbdad243dcc33834d1994705c494ec01bb8d4cea9042cb81e64442b8fd07c Jul 22 04:24:39.423909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.423821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:39.424095 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.423917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:39.424095 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.424019 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:39.424234 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.424119 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:40.424097717 +0000 UTC m=+4.513376885 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:39.424234 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.424025 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:39.424234 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.424186 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:24:40.42416927 +0000 UTC m=+4.513448437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:39.625614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.625580 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:39.625779 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.625734 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:24:39.625779 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.625756 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:24:39.625779 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.625769 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:39.625906 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.625850 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:40.625822393 +0000 UTC m=+4.715101545 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:39.838527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.838426 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 04:19:37 +0000 UTC" deadline="2028-03-09 16:20:01.682032779 +0000 UTC" Jul 22 04:24:39.838527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.838476 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14315h55m21.843560112s" Jul 22 04:24:39.887085 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:24:39.887043 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70691c50_ec06_4b34_966d_0730f0ebca13.slice/crio-b7a36dd35a40ee01ed9090e282b531fb1a21e5108b1706faaf1586747627449c WatchSource:0}: Error finding container b7a36dd35a40ee01ed9090e282b531fb1a21e5108b1706faaf1586747627449c: Status 404 returned error can't find the container with id b7a36dd35a40ee01ed9090e282b531fb1a21e5108b1706faaf1586747627449c Jul 22 04:24:39.925661 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.925633 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:39.925783 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:39.925758 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:39.938524 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.938492 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z9c8z" event={"ID":"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef","Type":"ContainerStarted","Data":"6631e6a159978920ac07f9692024857fb521be3a61c5fbe7724ea3722af58494"} Jul 22 04:24:39.939739 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.939711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerStarted","Data":"958a515b9e9fd8e518dcfff315698e8a8ad92f226d8e833908c279b6bf5b736c"} Jul 22 04:24:39.940895 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.940869 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vpdn" event={"ID":"bb43c786-c2c2-459e-bb69-7a661f7216ae","Type":"ContainerStarted","Data":"5e96d1ba401ed10c491b7aeb3831a2aa66e2b992c31800c5fe8d3f6619ee6310"} Jul 22 04:24:39.942032 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.942008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96kj6" event={"ID":"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3","Type":"ContainerStarted","Data":"fc87d5edf031e0bb5172319d4c1ed599f313a5773a10bdaa0864ce22b74cc10b"} Jul 22 04:24:39.943256 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.943230 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" event={"ID":"182c8bce-0ce8-4dc5-8a93-42cd57848c35","Type":"ContainerStarted","Data":"debf301303735708a3d60fba38af8094f493e33e800cc4f8bcc8a6c8b3834f34"} Jul 22 04:24:39.944524 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.944487 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"204dbdad243dcc33834d1994705c494ec01bb8d4cea9042cb81e64442b8fd07c"} Jul 22 04:24:39.945670 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.945649 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m29kf" event={"ID":"0cb9c999-df21-4553-b788-e757369a9963","Type":"ContainerStarted","Data":"9532f830cf5a6a44a77be54dd337e0fa87e9ac1952856ce1548322dd94c240ec"} Jul 22 04:24:39.946862 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.946834 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" event={"ID":"cc16b524-beac-46e2-a269-a082228fe97c","Type":"ContainerStarted","Data":"4e5405b2d81d2ba9a87d049c31f97bfd5054c252e5e7f8f21ed6ab3ccfd65038"} Jul 22 04:24:39.947947 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:39.947924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6bkv" event={"ID":"70691c50-ec06-4b34-966d-0730f0ebca13","Type":"ContainerStarted","Data":"b7a36dd35a40ee01ed9090e282b531fb1a21e5108b1706faaf1586747627449c"} Jul 22 04:24:40.431107 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.430570 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:40.431107 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.430628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:40.431107 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.430773 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:40.431107 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.430833 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:42.430813613 +0000 UTC m=+6.520092769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:40.431107 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.430952 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:40.431107 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.431024 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:24:42.431005297 +0000 UTC m=+6.520284461 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:40.633243 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.632583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:40.633243 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.632763 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:24:40.633243 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.632781 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:24:40.633243 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.632794 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:40.633243 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.632851 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:42.63283173 +0000 UTC m=+6.722110894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:40.928156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.927636 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:40.928156 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.927953 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:40.928156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.927992 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:40.928156 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:40.928088 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:40.963481 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.962852 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" event={"ID":"fe37cdaa3f7b18022e2f8cd5afc0ca1e","Type":"ContainerStarted","Data":"b11967e751a3573cd10aa97e4e29deeedf0857c8abf305168ae1cdcd32f50f79"} Jul 22 04:24:40.976577 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.976533 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" event={"ID":"c19c00cd70e4577ed80789f2afae047c","Type":"ContainerStarted","Data":"6257d372d99843ea1428e2e73e1bd95c5ec236bbf59c75295f05c0976d3bc04e"} Jul 22 04:24:40.992369 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:40.991564 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-182.ec2.internal" podStartSLOduration=3.991545751 podStartE2EDuration="3.991545751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:40.991204737 +0000 UTC m=+5.080483939" watchObservedRunningTime="2026-07-22 04:24:40.991545751 +0000 UTC m=+5.080824926" Jul 22 04:24:41.925673 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:41.925631 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:41.925860 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:41.925780 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:41.990617 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:41.990296 2566 generic.go:356] "Generic (PLEG): container finished" podID="fe37cdaa3f7b18022e2f8cd5afc0ca1e" containerID="b11967e751a3573cd10aa97e4e29deeedf0857c8abf305168ae1cdcd32f50f79" exitCode=0 Jul 22 04:24:41.990617 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:41.990393 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" event={"ID":"fe37cdaa3f7b18022e2f8cd5afc0ca1e","Type":"ContainerDied","Data":"b11967e751a3573cd10aa97e4e29deeedf0857c8abf305168ae1cdcd32f50f79"} Jul 22 04:24:42.451785 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:42.450830 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:42.451785 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:42.450941 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:42.451785 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.451136 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:42.451785 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.451200 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:24:46.451180451 +0000 UTC m=+10.540459606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:42.451785 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.451587 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:42.451785 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.451637 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:46.451620497 +0000 UTC m=+10.540899647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:42.652776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:42.652736 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:42.653510 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.652967 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:24:42.653510 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.652995 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:24:42.653510 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.653009 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:42.653510 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.653087 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:46.653050229 +0000 UTC m=+10.742329382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:42.926705 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:42.926170 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:42.926705 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.926298 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:42.927191 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:42.927045 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:42.927191 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:42.927154 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:43.001694 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:43.001172 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" event={"ID":"fe37cdaa3f7b18022e2f8cd5afc0ca1e","Type":"ContainerStarted","Data":"386a86e36bc30dbd081c08fc97999c29363ab4474370857d742649e7a2241943"} Jul 22 04:24:43.016955 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:43.016174 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-182.ec2.internal" podStartSLOduration=6.016155254 podStartE2EDuration="6.016155254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:43.014718101 +0000 UTC m=+7.103997328" watchObservedRunningTime="2026-07-22 04:24:43.016155254 +0000 UTC m=+7.105434428" Jul 22 04:24:43.925309 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:43.925259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:43.925477 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:43.925408 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:44.925717 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:44.925623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:44.926212 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:44.925764 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:44.926212 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:44.926141 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:44.926303 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:44.926255 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:45.925169 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:45.925132 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:45.925359 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:45.925275 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:46.485342 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:46.485307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:46.485798 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:46.485359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:46.485798 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.485479 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:46.485798 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.485544 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:54.485524584 +0000 UTC m=+18.574803737 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:46.485798 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.485480 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:46.485798 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.485621 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:24:54.485602558 +0000 UTC m=+18.574881709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:46.686938 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:46.686883 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:46.687132 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.687087 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:24:46.687132 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.687108 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:24:46.687132 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.687119 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:46.687299 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.687171 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:24:54.68715381 +0000 UTC m=+18.776432971 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:46.925827 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:46.925796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:46.925992 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.925917 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:46.925992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:46.925960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:46.926135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:46.926050 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:47.925817 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:47.925783 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:47.926291 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:47.925929 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:48.925098 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:48.925002 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:48.925098 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:48.925002 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:48.925330 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:48.925170 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:48.925330 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:48.925194 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:49.925700 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:49.925662 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:49.926157 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:49.925814 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:50.925380 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:50.925344 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:50.925560 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:50.925356 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:50.925560 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:50.925480 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:50.925560 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:50.925550 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:51.925136 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:51.925095 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:51.925589 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:51.925233 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:52.925032 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:52.924996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:52.925238 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:52.925146 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:52.925238 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:52.925196 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:52.925652 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:52.925322 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:53.925246 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:53.925214 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:53.925698 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:53.925356 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:54.548793 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:54.548755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:54.548980 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:54.548810 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:54.548980 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.548915 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:54.548980 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.548929 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:54.549174 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.548991 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:25:10.548975597 +0000 UTC m=+34.638254760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:24:54.549174 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.549014 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:10.549003706 +0000 UTC m=+34.638282859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:24:54.750340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:54.750304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:54.750539 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.750502 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:24:54.750539 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.750530 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:24:54.750539 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.750543 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:54.750748 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.750609 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:10.750590803 +0000 UTC m=+34.839869969 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:24:54.925680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:54.925649 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:54.925680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:54.925674 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:54.926240 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.925770 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:54.926240 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:54.925901 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:55.925160 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:55.925129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:55.925336 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:55.925250 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:56.925957 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:56.925921 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:56.926414 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:56.926023 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:56.926414 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:56.926089 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:56.926414 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:56.926196 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:57.925588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:57.925556 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:57.925747 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:57.925687 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:24:58.926328 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:58.925994 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:24:58.926934 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:58.926029 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:24:58.926934 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:58.926442 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:24:58.926934 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:58.926470 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:24:59.033520 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.033484 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z9c8z" event={"ID":"eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef","Type":"ContainerStarted","Data":"8e8aab32f4a1ae73d1b0b24ac585359d710c291857e60bf348808b6e0db3dc30"} Jul 22 04:24:59.035137 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.035109 2566 generic.go:356] "Generic (PLEG): container finished" podID="c091b74a-8aef-4cb2-8042-67b4e153cdb7" containerID="84da7cb8c9a69ed49b37124c34768eaf238119a5fb3ed35a60ab9e4250ec634b" exitCode=0 Jul 22 04:24:59.035275 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.035194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerDied","Data":"84da7cb8c9a69ed49b37124c34768eaf238119a5fb3ed35a60ab9e4250ec634b"} Jul 22 04:24:59.036673 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.036650 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7vpdn" event={"ID":"bb43c786-c2c2-459e-bb69-7a661f7216ae","Type":"ContainerStarted","Data":"9c60b31e710687ef7562516fc1e0d3c253a7f137fea0bf1f51150be0d57c3721"} Jul 22 04:24:59.039655 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.039621 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-96kj6" event={"ID":"3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3","Type":"ContainerStarted","Data":"8a1ca46a1e912976c7d58eecb61b6811e16cc5b0d1d12ccf95d720ff17f48bfe"} Jul 22 04:24:59.041341 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.041323 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" event={"ID":"182c8bce-0ce8-4dc5-8a93-42cd57848c35","Type":"ContainerStarted","Data":"4f3be87f3f870ec000a4034f77a2315b9903339f9e3aa44240f501c109ef447b"} Jul 22 04:24:59.043229 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.043208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"7d0e0a41c7a57d8fdbf70711b21d34f15394b8dcb5778b1347c0dec3e902a8aa"} Jul 22 04:24:59.043334 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.043271 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"1d8fde8c6f6532e7dce61731d4f886ef0f27bd4834f974624afbacc6dff31eec"} Jul 22 04:24:59.043334 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.043314 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"f8eef0edc1c0c1716e85c59406220bf55a3a4853234bbc7fd9234500ecab08dc"} Jul 22 04:24:59.044590 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.044572 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" event={"ID":"cc16b524-beac-46e2-a269-a082228fe97c","Type":"ContainerStarted","Data":"03fd4e916ecaadc83189df0508a5db467234a8114ec3766dc78578b146100a81"} Jul 22 04:24:59.045856 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.045826 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-s6bkv" event={"ID":"70691c50-ec06-4b34-966d-0730f0ebca13","Type":"ContainerStarted","Data":"36affdf3c2da827811bc436edaa61c6ab4bf3649260f47b0735babd178334150"} Jul 22 04:24:59.050462 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.050423 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z9c8z" podStartSLOduration=2.644601102 podStartE2EDuration="22.050407233s" totalImagesPullingTime="19.405806131s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:59.049362248 +0000 UTC m=+23.138641419" watchObservedRunningTime="2026-07-22 04:24:59.050407233 +0000 UTC m=+23.139686406" Jul 22 04:24:59.064857 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.064810 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m7ljf" podStartSLOduration=2.6548661239999998 podStartE2EDuration="22.064774998s" totalImagesPullingTime="19.409908874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:59.064294799 +0000 UTC m=+23.153573972" watchObservedRunningTime="2026-07-22 04:24:59.064774998 +0000 UTC m=+23.154054170" Jul 22 04:24:59.080530 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.080484 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-96kj6" podStartSLOduration=2.70505809 podStartE2EDuration="22.080473642s" totalImagesPullingTime="19.375415552s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:59.080360363 +0000 UTC m=+23.169639561" watchObservedRunningTime="2026-07-22 04:24:59.080473642 +0000 UTC m=+23.169752814" Jul 22 04:24:59.125729 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.125690 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-s6bkv" podStartSLOduration=3.491321948 podStartE2EDuration="22.12567468s" totalImagesPullingTime="18.634352732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:59.125476345 +0000 UTC m=+23.214755517" watchObservedRunningTime="2026-07-22 04:24:59.12567468 +0000 UTC m=+23.214953853" Jul 22 04:24:59.140554 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.140504 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7vpdn" podStartSLOduration=2.771576956 podStartE2EDuration="22.140484697s" totalImagesPullingTime="19.368907741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:24:59.1404835 +0000 UTC m=+23.229762672" watchObservedRunningTime="2026-07-22 04:24:59.140484697 +0000 UTC m=+23.229763865" Jul 22 04:24:59.925489 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:24:59.925459 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:24:59.925684 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:24:59.925574 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:00.052286 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.052249 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"456bdc5ec69315c2cf83ef2166fae55ec9b50a077cc814f3c6a96653e6d561e7"} Jul 22 04:25:00.052721 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.052351 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"d7aab559af6c11604fb830013c383ba09f316fb759fc0bda3fd77c198c6782da"} Jul 22 04:25:00.052721 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.052402 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"0fd9c87c7a20bda48082d13f865e4fc6f745a38c61eea986de95dfa60b28599e"} Jul 22 04:25:00.053976 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.053941 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m29kf" event={"ID":"0cb9c999-df21-4553-b788-e757369a9963","Type":"ContainerStarted","Data":"29a8cb70085a236914434562c896d26c21b0ceb4eaa5815c923a3d17d0f8bfa0"} Jul 22 04:25:00.068678 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.068610 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m29kf" podStartSLOduration=3.684325722 podStartE2EDuration="23.068595449s" totalImagesPullingTime="19.384269727s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:00.068209272 +0000 UTC m=+24.157488470" watchObservedRunningTime="2026-07-22 04:25:00.068595449 +0000 UTC m=+24.157874620" Jul 22 04:25:00.225803 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.225776 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 04:25:00.877111 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.876973 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T04:25:00.225795725Z","UUID":"8868446a-78cf-4412-a8d5-62e03c4a2280","Handler":null,"Name":"","Endpoint":""} Jul 22 04:25:00.878924 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.878898 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 04:25:00.879064 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.878946 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 04:25:00.925753 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.925718 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:00.925930 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.925725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:00.925930 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:00.925850 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:00.926047 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:00.925933 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:00.943833 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:00.943799 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:25:01.058864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:01.058812 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" event={"ID":"182c8bce-0ce8-4dc5-8a93-42cd57848c35","Type":"ContainerStarted","Data":"19ffd4e1e566fe4bd571c8d85a2f301367d6888081796470fd08f58cfc4f4821"} Jul 22 04:25:01.925028 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:01.924996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:01.925214 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:01.925139 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:02.064112 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:02.064060 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" event={"ID":"182c8bce-0ce8-4dc5-8a93-42cd57848c35","Type":"ContainerStarted","Data":"93efeac8419668e845ca457213f7165add3ed283116544c4dfe857d6e500c6ef"} Jul 22 04:25:02.071036 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:02.070989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"c496e5f88d737600c17c990bfc26ded29089f970faae857f7c55cafdaf1f33d2"} Jul 22 04:25:02.081433 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:02.081374 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l5fmk" podStartSLOduration=2.940320796 podStartE2EDuration="25.08135481s" totalImagesPullingTime="22.141034014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:02.081181022 +0000 UTC m=+26.170460217" watchObservedRunningTime="2026-07-22 04:25:02.08135481 +0000 UTC m=+26.170633982" Jul 22 04:25:02.925166 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:02.925126 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:02.925367 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:02.925126 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:02.925367 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:02.925265 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:02.925487 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:02.925375 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:03.424310 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:03.424279 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:25:03.425018 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:03.424862 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:25:03.925262 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:03.925054 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:03.925435 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:03.925347 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:04.076921 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.076843 2566 generic.go:356] "Generic (PLEG): container finished" podID="c091b74a-8aef-4cb2-8042-67b4e153cdb7" containerID="fdfded78e61bb13b1b5298d240fb0d8b5e9025f9b0da895d6629722fbff357f9" exitCode=0 Jul 22 04:25:04.077097 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.076916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerDied","Data":"fdfded78e61bb13b1b5298d240fb0d8b5e9025f9b0da895d6629722fbff357f9"} Jul 22 04:25:04.080664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.080642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" event={"ID":"e8971da1-28ee-4d18-bc40-8224326e12fc","Type":"ContainerStarted","Data":"1bd4a4da158d96c397fb61ee9e9ebf8e829db2165d0d001e799268c7840aee3f"} Jul 22 04:25:04.081552 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.081534 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-s6bkv" Jul 22 04:25:04.122041 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.121998 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" podStartSLOduration=7.699708192 podStartE2EDuration="27.121984259s" totalImagesPullingTime="19.422276067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:04.120639812 +0000 UTC m=+28.209918983" watchObservedRunningTime="2026-07-22 04:25:04.121984259 +0000 UTC m=+28.211263431" Jul 22 04:25:04.925602 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.925405 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:04.926464 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:04.925447 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:04.926464 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:04.925712 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:04.926464 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:04.925771 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:05.083204 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.083176 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:25:05.083406 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.083238 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:25:05.083406 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.083271 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:25:05.098791 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.098766 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:25:05.099260 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.099241 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:25:05.495161 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.495053 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jvssr"] Jul 22 04:25:05.495161 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.495155 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:05.495358 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:05.495231 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:05.501871 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.501836 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4fnp2"] Jul 22 04:25:05.501991 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.501955 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:05.502116 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:05.502089 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:05.502560 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.502543 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wlhqp"] Jul 22 04:25:05.502646 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:05.502611 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:05.502696 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:05.502680 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:06.086977 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:06.086942 2566 generic.go:356] "Generic (PLEG): container finished" podID="c091b74a-8aef-4cb2-8042-67b4e153cdb7" containerID="fc40bc9ef46232bcbcec0604b320cbf8967e5ce53898288df1c2f80615835043" exitCode=0 Jul 22 04:25:06.087539 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:06.087032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerDied","Data":"fc40bc9ef46232bcbcec0604b320cbf8967e5ce53898288df1c2f80615835043"} Jul 22 04:25:06.926419 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:06.926383 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:06.926603 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:06.926478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:06.926603 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:06.926524 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:06.926603 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:06.926571 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:06.926603 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:06.926570 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:06.926809 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:06.926664 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:08.094673 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:08.094638 2566 generic.go:356] "Generic (PLEG): container finished" podID="c091b74a-8aef-4cb2-8042-67b4e153cdb7" containerID="16c21852087872ddaccc6c2f50c4862280d195abcd7132c9d73699e1adbf54f9" exitCode=0 Jul 22 04:25:08.095053 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:08.094698 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerDied","Data":"16c21852087872ddaccc6c2f50c4862280d195abcd7132c9d73699e1adbf54f9"} Jul 22 04:25:08.925906 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:08.925861 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:08.925906 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:08.925875 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:08.926159 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:08.926021 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:08.926159 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:08.926100 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:08.926159 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:08.926117 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:08.926293 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:08.926196 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:10.563514 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:10.563480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:10.563973 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:10.563593 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:10.563973 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.563659 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:25:10.563973 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.563698 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 04:25:10.563973 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.563742 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:42.56371968 +0000 UTC m=+66.652998834 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 04:25:10.563973 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.563763 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret podName:a864c73c-bf76-41a9-a509-85976d29df5e nodeName:}" failed. No retries permitted until 2026-07-22 04:25:42.563753071 +0000 UTC m=+66.653032224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret") pod "global-pull-secret-syncer-jvssr" (UID: "a864c73c-bf76-41a9-a509-85976d29df5e") : object "kube-system"/"original-pull-secret" not registered Jul 22 04:25:10.765180 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:10.765142 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:10.765339 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.765319 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 04:25:10.765381 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.765341 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 04:25:10.765381 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.765351 2566 projected.go:196] Error preparing data for projected volume kube-api-access-rq4fr for pod openshift-network-diagnostics/network-check-target-wlhqp: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:25:10.765445 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.765407 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr podName:4fe21c2d-8788-4818-a21c-c63b3f42ae53 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:42.765391056 +0000 UTC m=+66.854670214 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rq4fr" (UniqueName: "kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr") pod "network-check-target-wlhqp" (UID: "4fe21c2d-8788-4818-a21c-c63b3f42ae53") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 04:25:10.925473 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:10.925369 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:10.925473 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:10.925419 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:10.925473 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:10.925373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:10.925764 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.925509 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jvssr" podUID="a864c73c-bf76-41a9-a509-85976d29df5e" Jul 22 04:25:10.925764 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.925609 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:25:10.925764 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:10.925667 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wlhqp" podUID="4fe21c2d-8788-4818-a21c-c63b3f42ae53" Jul 22 04:25:11.178983 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.178770 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-182.ec2.internal" event="NodeReady" Jul 22 04:25:11.179127 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.179055 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 04:25:11.214566 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.214531 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76fb9bc456-vqjgc"] Jul 22 04:25:11.216573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.216551 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.216975 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.216954 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm"] Jul 22 04:25:11.218558 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.218541 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.219122 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.219098 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 04:25:11.219229 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.219138 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 04:25:11.219229 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.219147 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 04:25:11.219229 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.219100 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xk8gb\"" Jul 22 04:25:11.220866 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.220844 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 04:25:11.221015 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.220970 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 04:25:11.221361 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.221341 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d4bx6\"" Jul 22 04:25:11.224925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.224904 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 04:25:11.229528 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.229510 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm"] Jul 22 04:25:11.230476 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.230236 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76fb9bc456-vqjgc"] Jul 22 04:25:11.237541 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.237408 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4c7tk"] Jul 22 04:25:11.240143 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.240120 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.240241 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.240170 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hdzxn"] Jul 22 04:25:11.241820 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.241801 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.255642 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.255621 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 04:25:11.255739 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.255621 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 04:25:11.255908 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.255892 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 04:25:11.256018 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.256000 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 04:25:11.256139 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.256115 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 04:25:11.256212 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.256138 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cjpdb\"" Jul 22 04:25:11.256468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.256451 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rp4tt\"" Jul 22 04:25:11.259387 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.259364 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4c7tk"] Jul 22 04:25:11.274568 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.274544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hdzxn"] Jul 22 04:25:11.369486 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-installation-pull-secrets\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369668 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-image-registry-private-configuration\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369668 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369668 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369545 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-ca-trust-extracted\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369668 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-tmp-dir\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsprs\" (UniqueName: \"kubernetes.io/projected/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-kube-api-access-xsprs\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-trusted-ca\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369737 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2k6d\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-kube-api-access-q2k6d\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369763 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-bound-sa-token\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369795 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1d48f3e-a7cf-4d48-9966-445390f29897-cert\") pod \"ingress-canary-hdzxn\" (UID: \"a1d48f3e-a7cf-4d48-9966-445390f29897\") " pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.369864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63329226-43a6-497b-86cd-bc6736e1edf5-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.370163 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369881 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-certificates\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.370163 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-config-volume\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.370163 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.370163 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.369935 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb7jh\" (UniqueName: \"kubernetes.io/projected/a1d48f3e-a7cf-4d48-9966-445390f29897-kube-api-access-xb7jh\") pod \"ingress-canary-hdzxn\" (UID: \"a1d48f3e-a7cf-4d48-9966-445390f29897\") " pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.470847 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470747 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-bound-sa-token\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.470847 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470793 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1d48f3e-a7cf-4d48-9966-445390f29897-cert\") pod \"ingress-canary-hdzxn\" (UID: \"a1d48f3e-a7cf-4d48-9966-445390f29897\") " pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.470847 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470849 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63329226-43a6-497b-86cd-bc6736e1edf5-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-certificates\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470895 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-config-volume\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7jh\" (UniqueName: \"kubernetes.io/projected/a1d48f3e-a7cf-4d48-9966-445390f29897-kube-api-access-xb7jh\") pod \"ingress-canary-hdzxn\" (UID: \"a1d48f3e-a7cf-4d48-9966-445390f29897\") " pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-installation-pull-secrets\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.470999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-image-registry-private-configuration\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-ca-trust-extracted\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471142 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-tmp-dir\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.471580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471159 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.471580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsprs\" (UniqueName: \"kubernetes.io/projected/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-kube-api-access-xsprs\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.471580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471224 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-trusted-ca\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2k6d\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-kube-api-access-q2k6d\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471539 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-config-volume\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.471580 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471576 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:11.471871 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471631 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-certificates\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.471871 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471652 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:11.971632434 +0000 UTC m=+36.060911613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:11.471871 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471735 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63329226-43a6-497b-86cd-bc6736e1edf5-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.471871 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.471810 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-tmp-dir\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.472106 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471896 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:11.472106 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471910 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:11.472106 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471944 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:11.472106 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471967 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:25:11.971950969 +0000 UTC m=+36.061230136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:11.472106 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.471995 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:11.971979779 +0000 UTC m=+36.061258944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:11.472372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.472193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-ca-trust-extracted\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.472548 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.472527 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-trusted-ca\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.476059 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.475994 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-installation-pull-secrets\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.476059 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.476019 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-image-registry-private-configuration\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.476059 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.476046 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1d48f3e-a7cf-4d48-9966-445390f29897-cert\") pod \"ingress-canary-hdzxn\" (UID: \"a1d48f3e-a7cf-4d48-9966-445390f29897\") " pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.479722 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.479698 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-bound-sa-token\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.480019 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.479997 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2k6d\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-kube-api-access-q2k6d\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.480299 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.480278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb7jh\" (UniqueName: \"kubernetes.io/projected/a1d48f3e-a7cf-4d48-9966-445390f29897-kube-api-access-xb7jh\") pod \"ingress-canary-hdzxn\" (UID: \"a1d48f3e-a7cf-4d48-9966-445390f29897\") " pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.480495 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.480475 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsprs\" (UniqueName: \"kubernetes.io/projected/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-kube-api-access-xsprs\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.555506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.555468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hdzxn" Jul 22 04:25:11.974544 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.974504 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.974560 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:11.974618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974676 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974728 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974735 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974752 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:12.974728944 +0000 UTC m=+37.064008093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974758 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974795 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:12.974779461 +0000 UTC m=+37.064058634 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:11.975135 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:11.974814 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:25:12.9748047 +0000 UTC m=+37.064083850 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:12.771862 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.771732 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hdzxn"] Jul 22 04:25:12.777552 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:25:12.777529 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d48f3e_a7cf_4d48_9966_445390f29897.slice/crio-6ad937c53c6122129f94c9968ffb03804327a323993d72a3132bc290fc517867 WatchSource:0}: Error finding container 6ad937c53c6122129f94c9968ffb03804327a323993d72a3132bc290fc517867: Status 404 returned error can't find the container with id 6ad937c53c6122129f94c9968ffb03804327a323993d72a3132bc290fc517867 Jul 22 04:25:12.925715 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.925677 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:12.925875 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.925680 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:12.925875 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.925680 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:12.928420 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.928396 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s9hfn\"" Jul 22 04:25:12.928548 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.928449 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 04:25:12.928915 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.928901 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 04:25:12.929014 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.928992 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h8l96\"" Jul 22 04:25:12.929152 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.929021 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 04:25:12.929152 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.929026 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 04:25:12.983660 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.983627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.983688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:12.983713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983787 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983801 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983811 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983839 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983856 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:14.983840106 +0000 UTC m=+39.073119277 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983875 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:25:14.983864936 +0000 UTC m=+39.073144086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:12.984307 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:12.983889 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:14.983881754 +0000 UTC m=+39.073160910 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:13.107391 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:13.107355 2566 generic.go:356] "Generic (PLEG): container finished" podID="c091b74a-8aef-4cb2-8042-67b4e153cdb7" containerID="e5b9efac3dda5197d6bd350fff79aa31d4ef38ce536914d6c3f2aa54580612ad" exitCode=0 Jul 22 04:25:13.107549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:13.107433 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerDied","Data":"e5b9efac3dda5197d6bd350fff79aa31d4ef38ce536914d6c3f2aa54580612ad"} Jul 22 04:25:13.108576 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:13.108552 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hdzxn" event={"ID":"a1d48f3e-a7cf-4d48-9966-445390f29897","Type":"ContainerStarted","Data":"6ad937c53c6122129f94c9968ffb03804327a323993d72a3132bc290fc517867"} Jul 22 04:25:14.114670 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:14.114635 2566 generic.go:356] "Generic (PLEG): container finished" podID="c091b74a-8aef-4cb2-8042-67b4e153cdb7" containerID="8ad9fdbffed558b43e79f0929f4093836545adbdc1059e268aab56b6f8f798c9" exitCode=0 Jul 22 04:25:14.115206 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:14.114684 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerDied","Data":"8ad9fdbffed558b43e79f0929f4093836545adbdc1059e268aab56b6f8f798c9"} Jul 22 04:25:15.002584 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:15.002555 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:15.002727 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:15.002605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:15.002727 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:15.002688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:15.002727 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002703 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:15.002877 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002759 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:19.002743322 +0000 UTC m=+43.092022472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:15.002877 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002779 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:15.002877 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002780 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:15.002877 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002802 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:15.002877 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002831 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:19.002818855 +0000 UTC m=+43.092098008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:15.002877 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:15.002854 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:25:19.002847846 +0000 UTC m=+43.092126996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:15.119921 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:15.119858 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-crnp4" event={"ID":"c091b74a-8aef-4cb2-8042-67b4e153cdb7","Type":"ContainerStarted","Data":"b55e67fb457e160b3df171565a57137049d86ca7c89c5c87c906c5edc5f3151a"} Jul 22 04:25:15.142470 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:15.142422 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-crnp4" podStartSLOduration=7.397381415 podStartE2EDuration="38.142407597s" totalImagesPullingTime="30.745026182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:15.141507853 +0000 UTC m=+39.230787040" watchObservedRunningTime="2026-07-22 04:25:15.142407597 +0000 UTC m=+39.231686769" Jul 22 04:25:16.124119 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:16.123857 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hdzxn" event={"ID":"a1d48f3e-a7cf-4d48-9966-445390f29897","Type":"ContainerStarted","Data":"785c6dfb2d6e6cbb89b25637e26c42f3760918d330d1a87bff0b01e7e705307f"} Jul 22 04:25:16.138372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:16.138329 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hdzxn" podStartSLOduration=2.731465631 podStartE2EDuration="5.138312807s" totalImagesPullingTime="2.406847176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:16.137719313 +0000 UTC m=+40.226998500" watchObservedRunningTime="2026-07-22 04:25:16.138312807 +0000 UTC m=+40.227591979" Jul 22 04:25:19.034955 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:19.034917 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:19.034982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:19.035005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035082 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035109 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035120 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035147 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:27.035130986 +0000 UTC m=+51.124410137 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035162 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:25:27.035155343 +0000 UTC m=+51.124434494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035162 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:19.035380 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:19.035216 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:27.035198609 +0000 UTC m=+51.124477766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:27.094358 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:27.094316 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:27.094366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:27.094421 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094462 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094537 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:43.094520639 +0000 UTC m=+67.183799790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094548 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094551 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094568 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094610 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:25:43.094594118 +0000 UTC m=+67.183873286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:27.094886 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:27.094628 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:25:43.09461917 +0000 UTC m=+67.183898321 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:37.110042 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:37.110005 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qnb42" Jul 22 04:25:42.609303 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.609260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:42.609790 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.609317 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:25:42.611922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.611903 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 04:25:42.611996 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.611951 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 04:25:42.619833 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:42.619800 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 04:25:42.620249 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:42.620218 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:26:46.619865288 +0000 UTC m=+130.709144449 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : secret "metrics-daemon-secret" not found Jul 22 04:25:42.626455 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.626425 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/a864c73c-bf76-41a9-a509-85976d29df5e-original-pull-secret\") pod \"global-pull-secret-syncer-jvssr\" (UID: \"a864c73c-bf76-41a9-a509-85976d29df5e\") " pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:42.635458 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.635431 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jvssr" Jul 22 04:25:42.764659 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.764625 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jvssr"] Jul 22 04:25:42.768514 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:25:42.768488 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda864c73c_bf76_41a9_a509_85976d29df5e.slice/crio-74174dff042a50ff2f2a66899f4b4d415001ff7869066ea928e9f1a9f99ab174 WatchSource:0}: Error finding container 74174dff042a50ff2f2a66899f4b4d415001ff7869066ea928e9f1a9f99ab174: Status 404 returned error can't find the container with id 74174dff042a50ff2f2a66899f4b4d415001ff7869066ea928e9f1a9f99ab174 Jul 22 04:25:42.811885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.811851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:42.814548 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.814524 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 04:25:42.824209 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.824183 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 04:25:42.835334 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.835307 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq4fr\" (UniqueName: \"kubernetes.io/projected/4fe21c2d-8788-4818-a21c-c63b3f42ae53-kube-api-access-rq4fr\") pod \"network-check-target-wlhqp\" (UID: \"4fe21c2d-8788-4818-a21c-c63b3f42ae53\") " pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:42.944024 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.943992 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h8l96\"" Jul 22 04:25:42.952001 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:42.951962 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:43.078922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:43.078889 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wlhqp"] Jul 22 04:25:43.081699 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:25:43.081670 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe21c2d_8788_4818_a21c_c63b3f42ae53.slice/crio-5041ddac0468b49716aba87d94598d9eaa3556a03977931a5fdf0076ce24dcca WatchSource:0}: Error finding container 5041ddac0468b49716aba87d94598d9eaa3556a03977931a5fdf0076ce24dcca: Status 404 returned error can't find the container with id 5041ddac0468b49716aba87d94598d9eaa3556a03977931a5fdf0076ce24dcca Jul 22 04:25:43.114114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:43.114047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:25:43.114318 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:43.114148 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:25:43.114318 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:43.114183 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:25:43.114318 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114206 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:25:43.114318 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114274 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:26:15.114256383 +0000 UTC m=+99.203535534 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:25:43.114318 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114290 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:25:43.114520 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114293 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:25:43.114520 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114342 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:26:15.114329329 +0000 UTC m=+99.203608480 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:25:43.114520 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114350 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:25:43.114520 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:25:43.114394 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:26:15.114381343 +0000 UTC m=+99.203660509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:25:43.184448 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:43.184408 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wlhqp" event={"ID":"4fe21c2d-8788-4818-a21c-c63b3f42ae53","Type":"ContainerStarted","Data":"5041ddac0468b49716aba87d94598d9eaa3556a03977931a5fdf0076ce24dcca"} Jul 22 04:25:43.185495 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:43.185466 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jvssr" event={"ID":"a864c73c-bf76-41a9-a509-85976d29df5e","Type":"ContainerStarted","Data":"74174dff042a50ff2f2a66899f4b4d415001ff7869066ea928e9f1a9f99ab174"} Jul 22 04:25:48.201120 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:48.201086 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wlhqp" event={"ID":"4fe21c2d-8788-4818-a21c-c63b3f42ae53","Type":"ContainerStarted","Data":"288ba86e1d8f94aad8d3b5d68ec213ee765759a1f7047b8a075aec671d9cfe05"} Jul 22 04:25:48.201647 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:48.201215 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:25:48.202420 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:48.202401 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jvssr" event={"ID":"a864c73c-bf76-41a9-a509-85976d29df5e","Type":"ContainerStarted","Data":"b3a645d8998f722a0fb9e37d832262ad06dd62b417cbb47f03e22f5d397f42e6"} Jul 22 04:25:48.216970 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:48.216928 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wlhqp" podStartSLOduration=66.637463906 podStartE2EDuration="1m11.216915368s" totalImagesPullingTime="4.579451462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:48.215943212 +0000 UTC m=+72.305222383" watchObservedRunningTime="2026-07-22 04:25:48.216915368 +0000 UTC m=+72.306194540" Jul 22 04:25:48.229554 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:25:48.229511 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jvssr" podStartSLOduration=66.332991834 podStartE2EDuration="1m11.229501078s" totalImagesPullingTime="4.896509244s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:25:48.228848476 +0000 UTC m=+72.318127648" watchObservedRunningTime="2026-07-22 04:25:48.229501078 +0000 UTC m=+72.318780247" Jul 22 04:26:15.152339 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:26:15.152282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:26:15.152339 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:26:15.152345 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:26:15.152383 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152451 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152471 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152476 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152489 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152531 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:27:19.152515461 +0000 UTC m=+163.241794625 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152552 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:27:19.152537009 +0000 UTC m=+163.241816162 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:26:15.152790 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:15.152571 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:27:19.152561773 +0000 UTC m=+163.241840926 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:26:19.207944 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:26:19.207912 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wlhqp" Jul 22 04:26:46.672895 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:26:46.672836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:26:46.673406 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:46.672989 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 04:26:46.673406 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:26:46.673060 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:48.673042061 +0000 UTC m=+252.762321215 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : secret "metrics-daemon-secret" not found Jul 22 04:27:14.228893 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:14.228836 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" Jul 22 04:27:14.241250 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:14.241218 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" podUID="63329226-43a6-497b-86cd-bc6736e1edf5" Jul 22 04:27:14.250396 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:14.250358 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4c7tk" podUID="c1d161be-db7d-4a6b-a1e1-7b54fdb04260" Jul 22 04:27:14.398734 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:14.398705 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4c7tk" Jul 22 04:27:14.398905 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:14.398706 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:27:14.398905 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:14.398707 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:27:15.946268 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:15.946220 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:27:19.206893 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:19.206861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:19.206915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.206997 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.207014 2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.207054 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls podName:c1d161be-db7d-4a6b-a1e1-7b54fdb04260 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:21.207039761 +0000 UTC m=+285.296318911 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls") pod "dns-default-4c7tk" (UID: "c1d161be-db7d-4a6b-a1e1-7b54fdb04260") : secret "dns-default-metrics-tls" not found Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:19.207091 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.207112 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert podName:63329226-43a6-497b-86cd-bc6736e1edf5 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:21.207093612 +0000 UTC m=+285.296372763 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-4s6vm" (UID: "63329226-43a6-497b-86cd-bc6736e1edf5") : secret "networking-console-plugin-cert" not found Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.207145 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.207154 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76fb9bc456-vqjgc: secret "image-registry-tls" not found Jul 22 04:27:19.207360 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:27:19.207186 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls podName:a1c5bcfb-34cc-42dc-9479-f0a116b58a1c nodeName:}" failed. No retries permitted until 2026-07-22 04:29:21.207178296 +0000 UTC m=+285.296457445 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls") pod "image-registry-76fb9bc456-vqjgc" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c") : secret "image-registry-tls" not found Jul 22 04:27:27.925231 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:27:27.925146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:28:45.184369 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.184326 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw"] Jul 22 04:28:45.186389 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.186368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" Jul 22 04:28:45.190200 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.190181 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-wl89n\"" Jul 22 04:28:45.193340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.193321 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx"] Jul 22 04:28:45.197666 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.197625 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd"] Jul 22 04:28:45.197795 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.197773 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.200245 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.200220 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s"] Jul 22 04:28:45.200335 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.200313 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" Jul 22 04:28:45.200731 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.200711 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 04:28:45.201048 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.201023 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5847r\"" Jul 22 04:28:45.201152 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.201138 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 04:28:45.201198 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.201182 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:28:45.202169 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.202150 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw"] Jul 22 04:28:45.202297 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.202227 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.202556 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.202535 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 04:28:45.202823 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.202809 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:28:45.203101 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.203061 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h4n2h\"" Jul 22 04:28:45.205198 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.205173 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 04:28:45.205198 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.205177 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-2blh2\"" Jul 22 04:28:45.205412 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.205214 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 04:28:45.205412 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.205219 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:28:45.205642 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.205522 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 04:28:45.208116 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.208095 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd"] Jul 22 04:28:45.211891 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.211841 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s"] Jul 22 04:28:45.212594 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.212575 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx"] Jul 22 04:28:45.275265 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.275235 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd"] Jul 22 04:28:45.277047 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.277030 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.279903 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.279877 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 04:28:45.280269 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.280244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-t8cgw\"" Jul 22 04:28:45.280384 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.280311 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 04:28:45.280456 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.280399 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 04:28:45.281670 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.281653 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 04:28:45.291944 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.291919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d024499-091d-499f-8f5c-3673b0a5b833-config\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.292061 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.291969 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnjlg\" (UniqueName: \"kubernetes.io/projected/03e46822-7796-468c-91b6-e72697f0e4ea-kube-api-access-tnjlg\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.292061 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb786\" (UniqueName: \"kubernetes.io/projected/5d024499-091d-499f-8f5c-3673b0a5b833-kube-api-access-wb786\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.292196 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292099 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vr2b\" (UniqueName: \"kubernetes.io/projected/a9f079ae-df39-4aa7-bb6c-222749daaab1-kube-api-access-4vr2b\") pod \"volume-data-source-validator-78cf4c6f6-vwnmd\" (UID: \"a9f079ae-df39-4aa7-bb6c-222749daaab1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" Jul 22 04:28:45.292196 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292169 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.292300 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnhdq\" (UniqueName: \"kubernetes.io/projected/a3a6ec40-78ae-4911-91ea-0a1e5d320108-kube-api-access-vnhdq\") pod \"network-check-source-7bc5b6bf85-tb6cw\" (UID: \"a3a6ec40-78ae-4911-91ea-0a1e5d320108\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" Jul 22 04:28:45.292300 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292284 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d024499-091d-499f-8f5c-3673b0a5b833-serving-cert\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.292397 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292317 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d967v\" (UniqueName: \"kubernetes.io/projected/a83059b5-edec-4145-b705-6e0c80767a12-kube-api-access-d967v\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.292397 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/03e46822-7796-468c-91b6-e72697f0e4ea-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.292490 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.292395 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.293179 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.293161 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd"] Jul 22 04:28:45.393588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393551 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d024499-091d-499f-8f5c-3673b0a5b833-serving-cert\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.393588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d967v\" (UniqueName: \"kubernetes.io/projected/a83059b5-edec-4145-b705-6e0c80767a12-kube-api-access-d967v\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.393853 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393615 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/03e46822-7796-468c-91b6-e72697f0e4ea-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.393853 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393649 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.393853 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d024499-091d-499f-8f5c-3673b0a5b833-config\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.393853 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.393763 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 04:28:45.393853 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnjlg\" (UniqueName: \"kubernetes.io/projected/03e46822-7796-468c-91b6-e72697f0e4ea-kube-api-access-tnjlg\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.393853 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.393846 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls podName:a83059b5-edec-4145-b705-6e0c80767a12 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:45.8938246 +0000 UTC m=+249.983103751 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tjcvx" (UID: "a83059b5-edec-4145-b705-6e0c80767a12") : secret "samples-operator-tls" not found Jul 22 04:28:45.394320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393893 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb786\" (UniqueName: \"kubernetes.io/projected/5d024499-091d-499f-8f5c-3673b0a5b833-kube-api-access-wb786\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.394320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393926 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vr2b\" (UniqueName: \"kubernetes.io/projected/a9f079ae-df39-4aa7-bb6c-222749daaab1-kube-api-access-4vr2b\") pod \"volume-data-source-validator-78cf4c6f6-vwnmd\" (UID: \"a9f079ae-df39-4aa7-bb6c-222749daaab1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" Jul 22 04:28:45.394320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.394320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.393998 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnhdq\" (UniqueName: \"kubernetes.io/projected/a3a6ec40-78ae-4911-91ea-0a1e5d320108-kube-api-access-vnhdq\") pod \"network-check-source-7bc5b6bf85-tb6cw\" (UID: \"a3a6ec40-78ae-4911-91ea-0a1e5d320108\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" Jul 22 04:28:45.394320 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.394130 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:45.394320 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.394213 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls podName:03e46822-7796-468c-91b6-e72697f0e4ea nodeName:}" failed. No retries permitted until 2026-07-22 04:28:45.894195895 +0000 UTC m=+249.983475051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-46mcd" (UID: "03e46822-7796-468c-91b6-e72697f0e4ea") : secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:45.394511 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.394429 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5d024499-091d-499f-8f5c-3673b0a5b833-config\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.394511 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.394497 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/03e46822-7796-468c-91b6-e72697f0e4ea-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.395912 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.395892 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5d024499-091d-499f-8f5c-3673b0a5b833-serving-cert\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.405762 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.405737 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb786\" (UniqueName: \"kubernetes.io/projected/5d024499-091d-499f-8f5c-3673b0a5b833-kube-api-access-wb786\") pod \"service-ca-operator-5bcbc94c78-thb4s\" (UID: \"5d024499-091d-499f-8f5c-3673b0a5b833\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.405887 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.405738 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vr2b\" (UniqueName: \"kubernetes.io/projected/a9f079ae-df39-4aa7-bb6c-222749daaab1-kube-api-access-4vr2b\") pod \"volume-data-source-validator-78cf4c6f6-vwnmd\" (UID: \"a9f079ae-df39-4aa7-bb6c-222749daaab1\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" Jul 22 04:28:45.406230 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.406207 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnhdq\" (UniqueName: \"kubernetes.io/projected/a3a6ec40-78ae-4911-91ea-0a1e5d320108-kube-api-access-vnhdq\") pod \"network-check-source-7bc5b6bf85-tb6cw\" (UID: \"a3a6ec40-78ae-4911-91ea-0a1e5d320108\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" Jul 22 04:28:45.406352 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.406335 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnjlg\" (UniqueName: \"kubernetes.io/projected/03e46822-7796-468c-91b6-e72697f0e4ea-kube-api-access-tnjlg\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.406352 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.406345 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d967v\" (UniqueName: \"kubernetes.io/projected/a83059b5-edec-4145-b705-6e0c80767a12-kube-api-access-d967v\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.499525 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.499435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" Jul 22 04:28:45.517105 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.517061 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" Jul 22 04:28:45.522094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.522055 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" Jul 22 04:28:45.636738 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.636589 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw"] Jul 22 04:28:45.639664 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:28:45.639625 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a6ec40_78ae_4911_91ea_0a1e5d320108.slice/crio-a094dbc2db767e4ff772130a762b8017bf9f899472451cf30a2696ae7f7a7917 WatchSource:0}: Error finding container a094dbc2db767e4ff772130a762b8017bf9f899472451cf30a2696ae7f7a7917: Status 404 returned error can't find the container with id a094dbc2db767e4ff772130a762b8017bf9f899472451cf30a2696ae7f7a7917 Jul 22 04:28:45.653285 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.653261 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd"] Jul 22 04:28:45.657294 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:28:45.657256 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9f079ae_df39_4aa7_bb6c_222749daaab1.slice/crio-18b65f67c7a689afcc3446471ee0bdbcf92145f3a0a90db521eed7be1d932551 WatchSource:0}: Error finding container 18b65f67c7a689afcc3446471ee0bdbcf92145f3a0a90db521eed7be1d932551: Status 404 returned error can't find the container with id 18b65f67c7a689afcc3446471ee0bdbcf92145f3a0a90db521eed7be1d932551 Jul 22 04:28:45.676285 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.676262 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s"] Jul 22 04:28:45.680090 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:28:45.680046 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d024499_091d_499f_8f5c_3673b0a5b833.slice/crio-0e793d79fc5f124996e36b1e91be3b2b6d6fc7f1d4c84085fa94e6bc5a3605f8 WatchSource:0}: Error finding container 0e793d79fc5f124996e36b1e91be3b2b6d6fc7f1d4c84085fa94e6bc5a3605f8: Status 404 returned error can't find the container with id 0e793d79fc5f124996e36b1e91be3b2b6d6fc7f1d4c84085fa94e6bc5a3605f8 Jul 22 04:28:45.899050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.899011 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:45.899264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:45.899101 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:45.899264 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.899187 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:45.899264 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.899234 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 04:28:45.899441 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.899273 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls podName:03e46822-7796-468c-91b6-e72697f0e4ea nodeName:}" failed. No retries permitted until 2026-07-22 04:28:46.899247384 +0000 UTC m=+250.988526536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-46mcd" (UID: "03e46822-7796-468c-91b6-e72697f0e4ea") : secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:45.899441 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:45.899292 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls podName:a83059b5-edec-4145-b705-6e0c80767a12 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:46.899283511 +0000 UTC m=+250.988562661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tjcvx" (UID: "a83059b5-edec-4145-b705-6e0c80767a12") : secret "samples-operator-tls" not found Jul 22 04:28:46.614692 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.614642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" event={"ID":"5d024499-091d-499f-8f5c-3673b0a5b833","Type":"ContainerStarted","Data":"0e793d79fc5f124996e36b1e91be3b2b6d6fc7f1d4c84085fa94e6bc5a3605f8"} Jul 22 04:28:46.616775 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.616728 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" event={"ID":"a3a6ec40-78ae-4911-91ea-0a1e5d320108","Type":"ContainerStarted","Data":"7dfc4b6930583cb3394574b0ac82282c1a0e43516b38b839f7de5f43b5c0ea84"} Jul 22 04:28:46.616775 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.616777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" event={"ID":"a3a6ec40-78ae-4911-91ea-0a1e5d320108","Type":"ContainerStarted","Data":"a094dbc2db767e4ff772130a762b8017bf9f899472451cf30a2696ae7f7a7917"} Jul 22 04:28:46.618279 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.618256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" event={"ID":"a9f079ae-df39-4aa7-bb6c-222749daaab1","Type":"ContainerStarted","Data":"18b65f67c7a689afcc3446471ee0bdbcf92145f3a0a90db521eed7be1d932551"} Jul 22 04:28:46.633879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.633831 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-tb6cw" podStartSLOduration=1.633814549 podStartE2EDuration="1.633814549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:28:46.632932966 +0000 UTC m=+250.722212157" watchObservedRunningTime="2026-07-22 04:28:46.633814549 +0000 UTC m=+250.723093722" Jul 22 04:28:46.906775 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.906709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:46.906894 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:46.906846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:46.906894 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:46.906876 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:46.907009 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:46.906945 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls podName:03e46822-7796-468c-91b6-e72697f0e4ea nodeName:}" failed. No retries permitted until 2026-07-22 04:28:48.906925306 +0000 UTC m=+252.996204472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-46mcd" (UID: "03e46822-7796-468c-91b6-e72697f0e4ea") : secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:46.907009 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:46.906984 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 04:28:46.907145 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:46.907036 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls podName:a83059b5-edec-4145-b705-6e0c80767a12 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:48.907020757 +0000 UTC m=+252.996299913 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tjcvx" (UID: "a83059b5-edec-4145-b705-6e0c80767a12") : secret "samples-operator-tls" not found Jul 22 04:28:47.622840 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:47.622798 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" event={"ID":"a9f079ae-df39-4aa7-bb6c-222749daaab1","Type":"ContainerStarted","Data":"29bffaec2e1ea8283a1576e4e9333f0ca82f8d59663c095eb993411c6773ccf7"} Jul 22 04:28:47.624235 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:47.624208 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" event={"ID":"5d024499-091d-499f-8f5c-3673b0a5b833","Type":"ContainerStarted","Data":"43725464410d68a336ae9561ad98258280ad0bd2179559b51bae1b6537ee0052"} Jul 22 04:28:47.636015 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:47.635961 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-vwnmd" podStartSLOduration=1.441475902 podStartE2EDuration="2.635941328s" totalImagesPullingTime="1.194465426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:28:47.635842497 +0000 UTC m=+251.725121667" watchObservedRunningTime="2026-07-22 04:28:47.635941328 +0000 UTC m=+251.725220538" Jul 22 04:28:47.648938 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:47.648887 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" podStartSLOduration=0.79546299 podStartE2EDuration="2.64887087s" totalImagesPullingTime="1.85340788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:28:47.647929088 +0000 UTC m=+251.737208283" watchObservedRunningTime="2026-07-22 04:28:47.64887087 +0000 UTC m=+251.738150042" Jul 22 04:28:48.720218 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:48.720169 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:28:48.720649 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:48.720304 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 22 04:28:48.720649 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:48.720377 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs podName:256b8fe7-db10-4e43-9d5c-d75acc517fb2 nodeName:}" failed. No retries permitted until 2026-07-22 04:30:50.720356837 +0000 UTC m=+374.809635986 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs") pod "network-metrics-daemon-4fnp2" (UID: "256b8fe7-db10-4e43-9d5c-d75acc517fb2") : secret "metrics-daemon-secret" not found Jul 22 04:28:48.921323 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:48.921280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:48.921494 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:48.921368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:48.921494 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:48.921442 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:48.921494 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:48.921467 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 04:28:48.921606 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:48.921511 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls podName:a83059b5-edec-4145-b705-6e0c80767a12 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:52.921494616 +0000 UTC m=+257.010773778 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tjcvx" (UID: "a83059b5-edec-4145-b705-6e0c80767a12") : secret "samples-operator-tls" not found Jul 22 04:28:48.921606 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:48.921528 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls podName:03e46822-7796-468c-91b6-e72697f0e4ea nodeName:}" failed. No retries permitted until 2026-07-22 04:28:52.921519158 +0000 UTC m=+257.010798313 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-46mcd" (UID: "03e46822-7796-468c-91b6-e72697f0e4ea") : secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:49.541103 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.541057 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tnrwz"] Jul 22 04:28:49.544247 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.544232 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.546391 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.546359 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 04:28:49.546518 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.546371 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 04:28:49.546518 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.546371 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qzd8f\"" Jul 22 04:28:49.546518 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.546377 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 04:28:49.547229 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.547215 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 04:28:49.557559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.557539 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tnrwz"] Jul 22 04:28:49.625745 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.625708 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcpv4\" (UniqueName: \"kubernetes.io/projected/4c014aef-a873-4c15-8033-04ccd2990813-kube-api-access-lcpv4\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.625907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.625768 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.625907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.625853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c014aef-a873-4c15-8033-04ccd2990813-crio-socket\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.625907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.625897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.626000 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.625916 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c014aef-a873-4c15-8033-04ccd2990813-data-volume\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727038 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.726999 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c014aef-a873-4c15-8033-04ccd2990813-data-volume\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.727116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcpv4\" (UniqueName: \"kubernetes.io/projected/4c014aef-a873-4c15-8033-04ccd2990813-kube-api-access-lcpv4\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.727182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.727221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c014aef-a873-4c15-8033-04ccd2990813-crio-socket\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.727257 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:49.727330 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:49.727356 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.727337 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4c014aef-a873-4c15-8033-04ccd2990813-crio-socket\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:49.727412 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:50.227393352 +0000 UTC m=+254.316672502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : configmap "kube-rbac-proxy" not found Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.727420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4c014aef-a873-4c15-8033-04ccd2990813-data-volume\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:49.727468 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:49.727441 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:50.227424307 +0000 UTC m=+254.316703473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : secret "insights-runtime-extractor-tls" not found Jul 22 04:28:49.736035 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:49.736006 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcpv4\" (UniqueName: \"kubernetes.io/projected/4c014aef-a873-4c15-8033-04ccd2990813-kube-api-access-lcpv4\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:50.232056 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.232020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:50.232264 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:50.232145 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:28:50.232264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.232162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:50.232264 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:50.232205 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:51.232189479 +0000 UTC m=+255.321468629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : configmap "kube-rbac-proxy" not found Jul 22 04:28:50.232264 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:50.232254 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:28:50.232421 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:50.232296 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:51.232285127 +0000 UTC m=+255.321564277 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : secret "insights-runtime-extractor-tls" not found Jul 22 04:28:50.693517 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.693482 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh"] Jul 22 04:28:50.696504 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.696483 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" Jul 22 04:28:50.698887 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.698864 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 04:28:50.699003 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.698903 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 04:28:50.699094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.699016 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-x999s\"" Jul 22 04:28:50.707625 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.707576 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh"] Jul 22 04:28:50.737536 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.737504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjdv5\" (UniqueName: \"kubernetes.io/projected/6d218c1c-8cb0-4dee-a4f9-db19ce62723c-kube-api-access-gjdv5\") pod \"migrator-7cc9c7ffd7-ppxvh\" (UID: \"6d218c1c-8cb0-4dee-a4f9-db19ce62723c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" Jul 22 04:28:50.838089 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.838020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjdv5\" (UniqueName: \"kubernetes.io/projected/6d218c1c-8cb0-4dee-a4f9-db19ce62723c-kube-api-access-gjdv5\") pod \"migrator-7cc9c7ffd7-ppxvh\" (UID: \"6d218c1c-8cb0-4dee-a4f9-db19ce62723c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" Jul 22 04:28:50.845753 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:50.845724 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjdv5\" (UniqueName: \"kubernetes.io/projected/6d218c1c-8cb0-4dee-a4f9-db19ce62723c-kube-api-access-gjdv5\") pod \"migrator-7cc9c7ffd7-ppxvh\" (UID: \"6d218c1c-8cb0-4dee-a4f9-db19ce62723c\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" Jul 22 04:28:51.004823 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.004723 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" Jul 22 04:28:51.060215 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.060181 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-729xw"] Jul 22 04:28:51.063898 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.063872 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.067218 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.067185 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 04:28:51.067321 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.067262 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 04:28:51.067392 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.067193 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 04:28:51.067584 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.067475 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-c9h6b\"" Jul 22 04:28:51.067584 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.067576 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 04:28:51.072053 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.072008 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-729xw"] Jul 22 04:28:51.132017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.131983 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh"] Jul 22 04:28:51.134855 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:28:51.134828 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d218c1c_8cb0_4dee_a4f9_db19ce62723c.slice/crio-d2fde0ff944d9cc558fed2121192b543fed6adc7b0873fb66dec38acd9c710da WatchSource:0}: Error finding container d2fde0ff944d9cc558fed2121192b543fed6adc7b0873fb66dec38acd9c710da: Status 404 returned error can't find the container with id d2fde0ff944d9cc558fed2121192b543fed6adc7b0873fb66dec38acd9c710da Jul 22 04:28:51.141263 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.141244 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmkv9\" (UniqueName: \"kubernetes.io/projected/9df0eee8-c39c-470c-bf1d-665b41343dde-kube-api-access-rmkv9\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.141335 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.141286 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9df0eee8-c39c-470c-bf1d-665b41343dde-signing-key\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.141432 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.141413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9df0eee8-c39c-470c-bf1d-665b41343dde-signing-cabundle\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.242731 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.242694 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:51.242907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.242751 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9df0eee8-c39c-470c-bf1d-665b41343dde-signing-cabundle\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.242907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.242794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:51.242907 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:51.242809 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:28:51.242907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.242835 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmkv9\" (UniqueName: \"kubernetes.io/projected/9df0eee8-c39c-470c-bf1d-665b41343dde-kube-api-access-rmkv9\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.242907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.242876 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9df0eee8-c39c-470c-bf1d-665b41343dde-signing-key\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.243184 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:51.242953 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:28:51.243184 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:51.242988 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:53.242966482 +0000 UTC m=+257.332245632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : configmap "kube-rbac-proxy" not found Jul 22 04:28:51.243184 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:51.243023 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:53.243005537 +0000 UTC m=+257.332284693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : secret "insights-runtime-extractor-tls" not found Jul 22 04:28:51.243436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.243416 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9df0eee8-c39c-470c-bf1d-665b41343dde-signing-cabundle\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.245253 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.245230 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9df0eee8-c39c-470c-bf1d-665b41343dde-signing-key\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.252455 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.252434 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmkv9\" (UniqueName: \"kubernetes.io/projected/9df0eee8-c39c-470c-bf1d-665b41343dde-kube-api-access-rmkv9\") pod \"service-ca-58598695f7-729xw\" (UID: \"9df0eee8-c39c-470c-bf1d-665b41343dde\") " pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.376418 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.376322 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-729xw" Jul 22 04:28:51.499452 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.499423 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-729xw"] Jul 22 04:28:51.501804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:28:51.501773 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df0eee8_c39c_470c_bf1d_665b41343dde.slice/crio-386ba87406ef9ebc8d9fe9238dc207c1ffd3b0f0e44c58ec26e2f30d2ffb42fa WatchSource:0}: Error finding container 386ba87406ef9ebc8d9fe9238dc207c1ffd3b0f0e44c58ec26e2f30d2ffb42fa: Status 404 returned error can't find the container with id 386ba87406ef9ebc8d9fe9238dc207c1ffd3b0f0e44c58ec26e2f30d2ffb42fa Jul 22 04:28:51.516648 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.516603 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96kj6_3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3/dns-node-resolver/0.log" Jul 22 04:28:51.643574 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.643226 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-729xw" event={"ID":"9df0eee8-c39c-470c-bf1d-665b41343dde","Type":"ContainerStarted","Data":"10ddb5048a8cdf15b137ae887e0e73d5c85e812f8f23bd614a8b935815635ed7"} Jul 22 04:28:51.643574 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.643287 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-729xw" event={"ID":"9df0eee8-c39c-470c-bf1d-665b41343dde","Type":"ContainerStarted","Data":"386ba87406ef9ebc8d9fe9238dc207c1ffd3b0f0e44c58ec26e2f30d2ffb42fa"} Jul 22 04:28:51.644551 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.644519 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" event={"ID":"6d218c1c-8cb0-4dee-a4f9-db19ce62723c","Type":"ContainerStarted","Data":"d2fde0ff944d9cc558fed2121192b543fed6adc7b0873fb66dec38acd9c710da"} Jul 22 04:28:51.662693 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:51.662621 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-729xw" podStartSLOduration=0.662603474 podStartE2EDuration="662.603474ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:28:51.661635592 +0000 UTC m=+255.750914766" watchObservedRunningTime="2026-07-22 04:28:51.662603474 +0000 UTC m=+255.751882623" Jul 22 04:28:52.320609 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:52.320517 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vpdn_bb43c786-c2c2-459e-bb69-7a661f7216ae/node-ca/0.log" Jul 22 04:28:52.651112 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:52.650909 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" event={"ID":"6d218c1c-8cb0-4dee-a4f9-db19ce62723c","Type":"ContainerStarted","Data":"2f3e7ab4167efbb1fe3496ee07c243046dd3a7648d778647cf640783e62dd744"} Jul 22 04:28:52.651112 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:52.650970 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" event={"ID":"6d218c1c-8cb0-4dee-a4f9-db19ce62723c","Type":"ContainerStarted","Data":"97eddae977aac898717af56546f3ac956823960508a67c57899500d70f94f42e"} Jul 22 04:28:52.680656 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:52.680600 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-ppxvh" podStartSLOduration=1.777336167 podStartE2EDuration="2.680579558s" totalImagesPullingTime="903.243391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:28:52.679914045 +0000 UTC m=+256.769193215" watchObservedRunningTime="2026-07-22 04:28:52.680579558 +0000 UTC m=+256.769858732" Jul 22 04:28:52.957943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:52.957850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:28:52.957943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:52.957919 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:28:52.958204 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:52.958025 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:52.958204 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:52.958119 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls podName:03e46822-7796-468c-91b6-e72697f0e4ea nodeName:}" failed. No retries permitted until 2026-07-22 04:29:00.958099351 +0000 UTC m=+265.047378517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-46mcd" (UID: "03e46822-7796-468c-91b6-e72697f0e4ea") : secret "cluster-monitoring-operator-tls" not found Jul 22 04:28:52.958204 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:52.958031 2566 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 04:28:52.958204 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:52.958188 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls podName:a83059b5-edec-4145-b705-6e0c80767a12 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:00.958177702 +0000 UTC m=+265.047456858 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-tjcvx" (UID: "a83059b5-edec-4145-b705-6e0c80767a12") : secret "samples-operator-tls" not found Jul 22 04:28:53.125308 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:53.125281 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hdzxn_a1d48f3e-a7cf-4d48-9966-445390f29897/serve-healthcheck-canary/0.log" Jul 22 04:28:53.260353 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:53.260254 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:53.260353 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:53.260319 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:53.260561 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:53.260377 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:28:53.260561 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:53.260451 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:57.260433501 +0000 UTC m=+261.349712667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : configmap "kube-rbac-proxy" not found Jul 22 04:28:53.260561 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:53.260490 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:28:53.260561 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:53.260557 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:28:57.260539871 +0000 UTC m=+261.349819036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : secret "insights-runtime-extractor-tls" not found Jul 22 04:28:57.294793 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:57.294750 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:57.295277 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:28:57.294823 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:28:57.295277 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:57.294878 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 04:28:57.295277 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:57.294942 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:05.294927045 +0000 UTC m=+269.384206208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : configmap "kube-rbac-proxy" not found Jul 22 04:28:57.295277 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:57.295038 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 04:28:57.295277 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:28:57.295146 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls podName:4c014aef-a873-4c15-8033-04ccd2990813 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:05.295130102 +0000 UTC m=+269.384409271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tnrwz" (UID: "4c014aef-a873-4c15-8033-04ccd2990813") : secret "insights-runtime-extractor-tls" not found Jul 22 04:29:01.027412 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.027317 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:29:01.027412 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.027373 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:29:01.029798 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.029776 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a83059b5-edec-4145-b705-6e0c80767a12-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-tjcvx\" (UID: \"a83059b5-edec-4145-b705-6e0c80767a12\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:29:01.029865 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.029777 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/03e46822-7796-468c-91b6-e72697f0e4ea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-46mcd\" (UID: \"03e46822-7796-468c-91b6-e72697f0e4ea\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:29:01.108782 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.108748 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" Jul 22 04:29:01.185844 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.185813 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" Jul 22 04:29:01.233634 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.233598 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx"] Jul 22 04:29:01.313978 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.313954 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd"] Jul 22 04:29:01.316725 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:01.316701 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03e46822_7796_468c_91b6_e72697f0e4ea.slice/crio-da1de933ee92312751745929e81e8eff7b0f248e63cc3e29819790a65580a248 WatchSource:0}: Error finding container da1de933ee92312751745929e81e8eff7b0f248e63cc3e29819790a65580a248: Status 404 returned error can't find the container with id da1de933ee92312751745929e81e8eff7b0f248e63cc3e29819790a65580a248 Jul 22 04:29:01.679001 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.678964 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" event={"ID":"a83059b5-edec-4145-b705-6e0c80767a12","Type":"ContainerStarted","Data":"864c9641d6d1b50cac4260a9725baaad64aded5486d2384865e044303f654a54"} Jul 22 04:29:01.680185 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:01.680150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" event={"ID":"03e46822-7796-468c-91b6-e72697f0e4ea","Type":"ContainerStarted","Data":"da1de933ee92312751745929e81e8eff7b0f248e63cc3e29819790a65580a248"} Jul 22 04:29:03.689229 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:03.689186 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" event={"ID":"a83059b5-edec-4145-b705-6e0c80767a12","Type":"ContainerStarted","Data":"ef96610d333f9ff57ad5f40d0e9388197d4f6d24a77aa6aa729fcb0e8c6197ca"} Jul 22 04:29:03.689721 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:03.689252 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" event={"ID":"a83059b5-edec-4145-b705-6e0c80767a12","Type":"ContainerStarted","Data":"724b5d1ce6a190e8d059005d7eb3702aea404a192b8adde68bc2b3da110131b7"} Jul 22 04:29:03.690728 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:03.690695 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" event={"ID":"03e46822-7796-468c-91b6-e72697f0e4ea","Type":"ContainerStarted","Data":"43fbf32882f6033860e65976bf80aa14a6148df492c5b3e6d787ded3983ddaac"} Jul 22 04:29:03.707264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:03.707215 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-tjcvx" podStartSLOduration=16.616593037 podStartE2EDuration="18.707196763s" totalImagesPullingTime="2.090603726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:03.705994435 +0000 UTC m=+267.795273608" watchObservedRunningTime="2026-07-22 04:29:03.707196763 +0000 UTC m=+267.796475935" Jul 22 04:29:03.720212 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:03.720162 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-46mcd" podStartSLOduration=16.650562014 podStartE2EDuration="18.720148856s" totalImagesPullingTime="2.069586842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:03.719724619 +0000 UTC m=+267.809003824" watchObservedRunningTime="2026-07-22 04:29:03.720148856 +0000 UTC m=+267.809428027" Jul 22 04:29:05.368429 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.368379 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:29:05.368942 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.368447 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:29:05.369063 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.369038 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4c014aef-a873-4c15-8033-04ccd2990813-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:29:05.370793 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.370771 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4c014aef-a873-4c15-8033-04ccd2990813-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tnrwz\" (UID: \"4c014aef-a873-4c15-8033-04ccd2990813\") " pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:29:05.453260 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.453208 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tnrwz" Jul 22 04:29:05.581991 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.581949 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tnrwz"] Jul 22 04:29:05.585531 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:05.585505 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c014aef_a873_4c15_8033_04ccd2990813.slice/crio-8bad339ec5bbb877f262fe5713224b82ead1d26c220bde8e17b15d6850567814 WatchSource:0}: Error finding container 8bad339ec5bbb877f262fe5713224b82ead1d26c220bde8e17b15d6850567814: Status 404 returned error can't find the container with id 8bad339ec5bbb877f262fe5713224b82ead1d26c220bde8e17b15d6850567814 Jul 22 04:29:05.698123 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:05.698078 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnrwz" event={"ID":"4c014aef-a873-4c15-8033-04ccd2990813","Type":"ContainerStarted","Data":"8bad339ec5bbb877f262fe5713224b82ead1d26c220bde8e17b15d6850567814"} Jul 22 04:29:10.716238 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:10.716205 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnrwz" event={"ID":"4c014aef-a873-4c15-8033-04ccd2990813","Type":"ContainerStarted","Data":"62be5cbb71a6c49750e697646e81627b02d4387333aee4e4285858898251c79d"} Jul 22 04:29:14.614127 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.614090 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-nkzrk"] Jul 22 04:29:14.617506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.617485 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:14.619660 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.619588 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 04:29:14.620000 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.619984 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8xw97\"" Jul 22 04:29:14.620126 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.620105 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 04:29:14.628832 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.628806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-nkzrk"] Jul 22 04:29:14.732833 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.732795 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnrwz" event={"ID":"4c014aef-a873-4c15-8033-04ccd2990813","Type":"ContainerStarted","Data":"bafa8b585d3723e85356f09c504ee3c75e48733c481de8035c9a8798cab9ec25"} Jul 22 04:29:14.753688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.753652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b49wj\" (UniqueName: \"kubernetes.io/projected/ba7434a6-9f90-47d6-9451-261245769db1-kube-api-access-b49wj\") pod \"downloads-5bc67dcd46-nkzrk\" (UID: \"ba7434a6-9f90-47d6-9451-261245769db1\") " pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:14.854549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.854509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b49wj\" (UniqueName: \"kubernetes.io/projected/ba7434a6-9f90-47d6-9451-261245769db1-kube-api-access-b49wj\") pod \"downloads-5bc67dcd46-nkzrk\" (UID: \"ba7434a6-9f90-47d6-9451-261245769db1\") " pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:14.863909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.863869 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b49wj\" (UniqueName: \"kubernetes.io/projected/ba7434a6-9f90-47d6-9451-261245769db1-kube-api-access-b49wj\") pod \"downloads-5bc67dcd46-nkzrk\" (UID: \"ba7434a6-9f90-47d6-9451-261245769db1\") " pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:14.928827 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:14.928782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:15.069170 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:15.069140 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-nkzrk"] Jul 22 04:29:15.073206 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:15.073177 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba7434a6_9f90_47d6_9451_261245769db1.slice/crio-5c102484340c4384b89f26332e00d7cc5756240a48b109f46ff64b929a08a190 WatchSource:0}: Error finding container 5c102484340c4384b89f26332e00d7cc5756240a48b109f46ff64b929a08a190: Status 404 returned error can't find the container with id 5c102484340c4384b89f26332e00d7cc5756240a48b109f46ff64b929a08a190 Jul 22 04:29:15.737778 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:15.737731 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-nkzrk" event={"ID":"ba7434a6-9f90-47d6-9451-261245769db1","Type":"ContainerStarted","Data":"5c102484340c4384b89f26332e00d7cc5756240a48b109f46ff64b929a08a190"} Jul 22 04:29:17.399858 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:17.399815 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4c7tk" podUID="c1d161be-db7d-4a6b-a1e1-7b54fdb04260" Jul 22 04:29:17.399858 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:17.399847 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" podUID="63329226-43a6-497b-86cd-bc6736e1edf5" Jul 22 04:29:17.399858 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:17.399833 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" Jul 22 04:29:17.748910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:17.748803 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tnrwz" event={"ID":"4c014aef-a873-4c15-8033-04ccd2990813","Type":"ContainerStarted","Data":"a610985c7aa182c54c0d0e22a179bc755a6c871a3f4b26f7539d613c5e4f956d"} Jul 22 04:29:17.748910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:17.748853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:29:17.748910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:17.748874 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4c7tk" Jul 22 04:29:17.749220 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:17.749117 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:29:17.768510 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:17.768461 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tnrwz" podStartSLOduration=16.995457541 podStartE2EDuration="28.768446306s" totalImagesPullingTime="11.772988765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:28:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:17.767570759 +0000 UTC m=+281.856849952" watchObservedRunningTime="2026-07-22 04:29:17.768446306 +0000 UTC m=+281.857725478" Jul 22 04:29:21.218141 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.218104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:29:21.218558 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.218157 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:29:21.218558 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.218226 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:29:21.220786 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.220760 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c1d161be-db7d-4a6b-a1e1-7b54fdb04260-metrics-tls\") pod \"dns-default-4c7tk\" (UID: \"c1d161be-db7d-4a6b-a1e1-7b54fdb04260\") " pod="openshift-dns/dns-default-4c7tk" Jul 22 04:29:21.220910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.220791 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/63329226-43a6-497b-86cd-bc6736e1edf5-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-4s6vm\" (UID: \"63329226-43a6-497b-86cd-bc6736e1edf5\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:29:21.220910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.220851 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"image-registry-76fb9bc456-vqjgc\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:29:21.353200 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.353169 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xk8gb\"" Jul 22 04:29:21.353378 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.353246 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cjpdb\"" Jul 22 04:29:21.353378 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.353303 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-d4bx6\"" Jul 22 04:29:21.360363 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.360342 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:29:21.360492 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.360451 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4c7tk" Jul 22 04:29:21.360581 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.360493 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" Jul 22 04:29:21.522482 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.520822 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4c7tk"] Jul 22 04:29:21.528742 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:21.528670 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1d161be_db7d_4a6b_a1e1_7b54fdb04260.slice/crio-53e8981d8ed2f136c47e2833511875a16e1abbe317262e2347b048edc7e18e23 WatchSource:0}: Error finding container 53e8981d8ed2f136c47e2833511875a16e1abbe317262e2347b048edc7e18e23: Status 404 returned error can't find the container with id 53e8981d8ed2f136c47e2833511875a16e1abbe317262e2347b048edc7e18e23 Jul 22 04:29:21.751421 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.751142 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm"] Jul 22 04:29:21.754457 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:21.754386 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63329226_43a6_497b_86cd_bc6736e1edf5.slice/crio-1b050fce5be8951999a0f512f8c5bff4a194da9d8c593259ab55fe90de028377 WatchSource:0}: Error finding container 1b050fce5be8951999a0f512f8c5bff4a194da9d8c593259ab55fe90de028377: Status 404 returned error can't find the container with id 1b050fce5be8951999a0f512f8c5bff4a194da9d8c593259ab55fe90de028377 Jul 22 04:29:21.755266 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.755244 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76fb9bc456-vqjgc"] Jul 22 04:29:21.757805 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:21.757784 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c5bcfb_34cc_42dc_9479_f0a116b58a1c.slice/crio-d1b2a2210aa97fb7dc92cf817aeebaa1ae4bce54ca42a9af922c940ed936190a WatchSource:0}: Error finding container d1b2a2210aa97fb7dc92cf817aeebaa1ae4bce54ca42a9af922c940ed936190a: Status 404 returned error can't find the container with id d1b2a2210aa97fb7dc92cf817aeebaa1ae4bce54ca42a9af922c940ed936190a Jul 22 04:29:21.764332 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.764303 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4c7tk" event={"ID":"c1d161be-db7d-4a6b-a1e1-7b54fdb04260","Type":"ContainerStarted","Data":"53e8981d8ed2f136c47e2833511875a16e1abbe317262e2347b048edc7e18e23"} Jul 22 04:29:21.765697 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.765647 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" event={"ID":"63329226-43a6-497b-86cd-bc6736e1edf5","Type":"ContainerStarted","Data":"1b050fce5be8951999a0f512f8c5bff4a194da9d8c593259ab55fe90de028377"} Jul 22 04:29:21.766879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:21.766839 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" event={"ID":"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c","Type":"ContainerStarted","Data":"d1b2a2210aa97fb7dc92cf817aeebaa1ae4bce54ca42a9af922c940ed936190a"} Jul 22 04:29:22.772737 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:22.772663 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" event={"ID":"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c","Type":"ContainerStarted","Data":"88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e"} Jul 22 04:29:22.773234 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:22.772974 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:29:22.791497 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:22.791453 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" podStartSLOduration=263.791431858 podStartE2EDuration="4m23.791431858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:24:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:22.791361307 +0000 UTC m=+286.880640501" watchObservedRunningTime="2026-07-22 04:29:22.791431858 +0000 UTC m=+286.880711033" Jul 22 04:29:23.288095 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.288013 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg"] Jul 22 04:29:23.291728 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.291698 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.294537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.294381 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 22 04:29:23.294537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.294475 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-dck8t\"" Jul 22 04:29:23.294709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.294381 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 22 04:29:23.295144 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.294991 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 04:29:23.303755 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.303418 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg"] Jul 22 04:29:23.313043 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.313023 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-z7ln7"] Jul 22 04:29:23.316908 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.316892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.323022 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.321170 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 22 04:29:23.323022 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.322014 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 22 04:29:23.323022 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.322233 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 22 04:29:23.323022 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.322498 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-rdcwp\"" Jul 22 04:29:23.326010 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.324564 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7m876"] Jul 22 04:29:23.328131 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.328113 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-z7ln7"] Jul 22 04:29:23.328366 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.328314 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.333685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.333276 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 04:29:23.333685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.333408 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 04:29:23.333685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.333504 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-svvf9\"" Jul 22 04:29:23.334288 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.334266 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 04:29:23.339698 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.339698 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339676 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26688b8c-fde4-486e-8527-cf9795f8355b-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.339852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.339852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.339852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339770 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-554cn\" (UniqueName: \"kubernetes.io/projected/26688b8c-fde4-486e-8527-cf9795f8355b-kube-api-access-554cn\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.339852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339794 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8v9x\" (UniqueName: \"kubernetes.io/projected/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-kube-api-access-f8v9x\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.340051 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.340051 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.340051 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339930 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26688b8c-fde4-486e-8527-cf9795f8355b-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.340051 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.339954 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440320 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-metrics-client-ca\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440368 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26688b8c-fde4-486e-8527-cf9795f8355b-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440399 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-wtmp\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440460 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440498 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26688b8c-fde4-486e-8527-cf9795f8355b-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440524 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn2bl\" (UniqueName: \"kubernetes.io/projected/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-kube-api-access-kn2bl\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-root\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440572 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440600 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.440641 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-554cn\" (UniqueName: \"kubernetes.io/projected/26688b8c-fde4-486e-8527-cf9795f8355b-kube-api-access-554cn\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440692 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8v9x\" (UniqueName: \"kubernetes.io/projected/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-kube-api-access-f8v9x\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-textfile\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440739 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-sys\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440762 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440832 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.440861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.441138 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/26688b8c-fde4-486e-8527-cf9795f8355b-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:23.441183 2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 22 04:29:23.441312 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:23.441247 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-tls podName:65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:23.941228185 +0000 UTC m=+288.030507355 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-qlfrg" (UID: "65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931") : secret "openshift-state-metrics-tls" not found Jul 22 04:29:23.441856 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:23.441729 2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 22 04:29:23.441856 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:23.441775 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-tls podName:26688b8c-fde4-486e-8527-cf9795f8355b nodeName:}" failed. No retries permitted until 2026-07-22 04:29:23.941759765 +0000 UTC m=+288.031038921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-z7ln7" (UID: "26688b8c-fde4-486e-8527-cf9795f8355b") : secret "kube-state-metrics-tls" not found Jul 22 04:29:23.442587 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.441998 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.442587 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.442130 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/26688b8c-fde4-486e-8527-cf9795f8355b-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.442587 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.442541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.450183 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.450139 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.451004 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.450925 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.451713 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.451657 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8v9x\" (UniqueName: \"kubernetes.io/projected/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-kube-api-access-f8v9x\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.453610 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.453557 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-554cn\" (UniqueName: \"kubernetes.io/projected/26688b8c-fde4-486e-8527-cf9795f8355b-kube-api-access-554cn\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.541704 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541620 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-textfile\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541704 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541671 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-sys\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541704 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541701 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541958 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541958 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541768 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-sys\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541958 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541788 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-metrics-client-ca\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541958 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541839 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-wtmp\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.541958 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541929 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn2bl\" (UniqueName: \"kubernetes.io/projected/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-kube-api-access-kn2bl\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541961 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-textfile\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541965 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-root\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541991 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-wtmp\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.541993 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.542136 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-root\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:23.542141 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 04:29:23.542226 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:23.542198 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls podName:65a58f4e-fd6d-45c0-8d15-dcbe42d877d9 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:24.042181213 +0000 UTC m=+288.131460363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls") pod "node-exporter-7m876" (UID: "65a58f4e-fd6d-45c0-8d15-dcbe42d877d9") : secret "node-exporter-tls" not found Jul 22 04:29:23.543340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.542610 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-accelerators-collector-config\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.543340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.543014 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-metrics-client-ca\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.545952 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.545926 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.550647 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.550606 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn2bl\" (UniqueName: \"kubernetes.io/projected/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-kube-api-access-kn2bl\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:23.779995 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.779895 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4c7tk" event={"ID":"c1d161be-db7d-4a6b-a1e1-7b54fdb04260","Type":"ContainerStarted","Data":"1efdf3f61894d26b2d10d74b12d8a63d2c63b992d97e6362eb707332aa820cab"} Jul 22 04:29:23.782819 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.782737 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" event={"ID":"63329226-43a6-497b-86cd-bc6736e1edf5","Type":"ContainerStarted","Data":"25a8fcfe561bd9457cfac71e35a47c72b8fa6081f8cf62b221689da736eacad1"} Jul 22 04:29:23.800549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.800169 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-4s6vm" podStartSLOduration=278.938737874 podStartE2EDuration="4m40.800149258s" totalImagesPullingTime="1.861411384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:24:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:23.798262995 +0000 UTC m=+287.887542167" watchObservedRunningTime="2026-07-22 04:29:23.800149258 +0000 UTC m=+287.889428432" Jul 22 04:29:23.946305 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.946263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.946482 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.946327 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:23.948974 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.948950 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/26688b8c-fde4-486e-8527-cf9795f8355b-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-z7ln7\" (UID: \"26688b8c-fde4-486e-8527-cf9795f8355b\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:23.949114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:23.949090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-qlfrg\" (UID: \"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:24.047606 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.047571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:24.047773 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:24.047706 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 04:29:24.047773 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:24.047765 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls podName:65a58f4e-fd6d-45c0-8d15-dcbe42d877d9 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:25.047746653 +0000 UTC m=+289.137025809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls") pod "node-exporter-7m876" (UID: "65a58f4e-fd6d-45c0-8d15-dcbe42d877d9") : secret "node-exporter-tls" not found Jul 22 04:29:24.217151 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.217118 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" Jul 22 04:29:24.237613 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.237586 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" Jul 22 04:29:24.384829 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.384764 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg"] Jul 22 04:29:24.388506 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:24.388461 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e9d171_e5f7_4f44_a2e7_5f2bcc1c4931.slice/crio-0c66063564b2bc7e9e26bf49a0dfa3d32e09c18c0c8fdafe016f749e89bd3c36 WatchSource:0}: Error finding container 0c66063564b2bc7e9e26bf49a0dfa3d32e09c18c0c8fdafe016f749e89bd3c36: Status 404 returned error can't find the container with id 0c66063564b2bc7e9e26bf49a0dfa3d32e09c18c0c8fdafe016f749e89bd3c36 Jul 22 04:29:24.401479 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.401395 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-z7ln7"] Jul 22 04:29:24.405226 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:24.405202 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26688b8c_fde4_486e_8527_cf9795f8355b.slice/crio-d900f047fac88ea43014864e6e97e59fe0635a5df1a4e929076da918ce315258 WatchSource:0}: Error finding container d900f047fac88ea43014864e6e97e59fe0635a5df1a4e929076da918ce315258: Status 404 returned error can't find the container with id d900f047fac88ea43014864e6e97e59fe0635a5df1a4e929076da918ce315258 Jul 22 04:29:24.788840 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.788706 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4c7tk" event={"ID":"c1d161be-db7d-4a6b-a1e1-7b54fdb04260","Type":"ContainerStarted","Data":"20e4a3d55024aaef91c2b493b1459b4704afc93808b6f07bf086ede21568de16"} Jul 22 04:29:24.789293 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.788844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4c7tk" Jul 22 04:29:24.791028 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.790958 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" event={"ID":"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931","Type":"ContainerStarted","Data":"ee16efa2c5879785a94d52aed2bf11d489b7d22a07b97fa68ee9ccaf7a6c0ea0"} Jul 22 04:29:24.791028 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.791016 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" event={"ID":"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931","Type":"ContainerStarted","Data":"61b65e53004dcfbc4ad6684ae53769c26e76854931f9544350dcbcfe5342b1d9"} Jul 22 04:29:24.791240 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.791047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" event={"ID":"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931","Type":"ContainerStarted","Data":"0c66063564b2bc7e9e26bf49a0dfa3d32e09c18c0c8fdafe016f749e89bd3c36"} Jul 22 04:29:24.792589 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.792388 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" event={"ID":"26688b8c-fde4-486e-8527-cf9795f8355b","Type":"ContainerStarted","Data":"d900f047fac88ea43014864e6e97e59fe0635a5df1a4e929076da918ce315258"} Jul 22 04:29:24.805751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:24.805698 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4c7tk" podStartSLOduration=251.707484898 podStartE2EDuration="4m13.805681581s" totalImagesPullingTime="2.098196683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:24.805024556 +0000 UTC m=+288.894303821" watchObservedRunningTime="2026-07-22 04:29:24.805681581 +0000 UTC m=+288.894960756" Jul 22 04:29:25.057399 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:25.057318 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:25.060042 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:25.060002 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/65a58f4e-fd6d-45c0-8d15-dcbe42d877d9-node-exporter-tls\") pod \"node-exporter-7m876\" (UID: \"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9\") " pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:25.145047 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:25.145010 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7m876" Jul 22 04:29:26.286047 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.286012 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-8674fd69d-5t2vd"] Jul 22 04:29:26.290185 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.290142 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.293250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.293220 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lnt5q\"" Jul 22 04:29:26.293580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.293444 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 22 04:29:26.293580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.293461 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-j1bdcm7bscj7\"" Jul 22 04:29:26.293774 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.293754 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 22 04:29:26.293889 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.293801 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 22 04:29:26.293991 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.293952 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 22 04:29:26.294110 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.294036 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 22 04:29:26.302337 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.302315 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8674fd69d-5t2vd"] Jul 22 04:29:26.368786 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.368741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-tls\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.368940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.368814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.368940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.368842 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wph7v\" (UniqueName: \"kubernetes.io/projected/d7457d92-17c3-4353-9cd9-a88214127be4-kube-api-access-wph7v\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.368940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.368870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7457d92-17c3-4353-9cd9-a88214127be4-metrics-client-ca\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.368940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.368897 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.369188 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.368953 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.369188 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.369021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.369188 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.369043 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-grpc-tls\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.469704 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469665 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.469882 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wph7v\" (UniqueName: \"kubernetes.io/projected/d7457d92-17c3-4353-9cd9-a88214127be4-kube-api-access-wph7v\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.469882 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469772 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7457d92-17c3-4353-9cd9-a88214127be4-metrics-client-ca\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.469882 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.469882 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469845 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.470133 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.470133 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-grpc-tls\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.470133 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.469981 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-tls\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.471087 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.470873 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d7457d92-17c3-4353-9cd9-a88214127be4-metrics-client-ca\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.473311 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.473268 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-tls\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.473818 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.473777 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.474588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.474251 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.474588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.474556 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.475129 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.475105 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.475254 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.475234 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d7457d92-17c3-4353-9cd9-a88214127be4-secret-grpc-tls\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.477850 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.477816 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wph7v\" (UniqueName: \"kubernetes.io/projected/d7457d92-17c3-4353-9cd9-a88214127be4-kube-api-access-wph7v\") pod \"thanos-querier-8674fd69d-5t2vd\" (UID: \"d7457d92-17c3-4353-9cd9-a88214127be4\") " pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:26.602662 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:26.602580 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:27.578621 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.578581 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cdd7b794b-6qpfs"] Jul 22 04:29:27.582982 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.582960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.586333 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.586045 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 04:29:27.586333 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.586165 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 04:29:27.586333 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.586225 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 04:29:27.586333 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.586225 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 04:29:27.586333 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.586277 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 04:29:27.586579 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.586431 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-r5m4w\"" Jul 22 04:29:27.590378 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.590358 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 04:29:27.592244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.592199 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdd7b794b-6qpfs"] Jul 22 04:29:27.679177 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-serving-cert\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.679340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-trusted-ca-bundle\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.679340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679218 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-oauth-serving-cert\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.679340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679305 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-config\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.679513 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m4zp\" (UniqueName: \"kubernetes.io/projected/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-kube-api-access-5m4zp\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.679513 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-service-ca\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.679513 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.679452 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-oauth-config\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.698926 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.698886 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-56668596dd-4xm4j"] Jul 22 04:29:27.702360 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.702339 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.705395 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.705371 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 22 04:29:27.705506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.705412 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 22 04:29:27.705506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.705367 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-107vqov5elhj3\"" Jul 22 04:29:27.705506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.705459 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-b8ddm\"" Jul 22 04:29:27.705784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.705761 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 22 04:29:27.706102 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.706085 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 22 04:29:27.716476 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.716455 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56668596dd-4xm4j"] Jul 22 04:29:27.780883 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.780837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0e64a6d2-f1e8-4953-93a5-1beb6732d482-audit-log\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.780883 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.780879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-secret-metrics-server-client-certs\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.781156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.780913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-config\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.780938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m4zp\" (UniqueName: \"kubernetes.io/projected/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-kube-api-access-5m4zp\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781113 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-service-ca\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781149 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-oauth-config\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781367 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-secret-metrics-server-tls\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.781367 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781250 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0e64a6d2-f1e8-4953-93a5-1beb6732d482-metrics-server-audit-profiles\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.781367 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781286 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-serving-cert\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781517 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781395 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e64a6d2-f1e8-4953-93a5-1beb6732d482-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.781517 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781443 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-client-ca-bundle\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.781517 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781505 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-trusted-ca-bundle\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781671 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781534 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-oauth-serving-cert\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781671 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxw84\" (UniqueName: \"kubernetes.io/projected/0e64a6d2-f1e8-4953-93a5-1beb6732d482-kube-api-access-xxw84\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.781775 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781761 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-config\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.781938 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.781914 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-service-ca\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.782306 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.782283 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-oauth-serving-cert\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.782306 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.782303 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-trusted-ca-bundle\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.783949 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.783930 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-serving-cert\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.784036 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.783995 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-oauth-config\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.788880 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.788857 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m4zp\" (UniqueName: \"kubernetes.io/projected/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-kube-api-access-5m4zp\") pod \"console-7cdd7b794b-6qpfs\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:27.882588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxw84\" (UniqueName: \"kubernetes.io/projected/0e64a6d2-f1e8-4953-93a5-1beb6732d482-kube-api-access-xxw84\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.882588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0e64a6d2-f1e8-4953-93a5-1beb6732d482-audit-log\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.882825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882608 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-secret-metrics-server-client-certs\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.882825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-secret-metrics-server-tls\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.882825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0e64a6d2-f1e8-4953-93a5-1beb6732d482-metrics-server-audit-profiles\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.882825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e64a6d2-f1e8-4953-93a5-1beb6732d482-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.883018 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.882840 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-client-ca-bundle\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.883128 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.883098 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0e64a6d2-f1e8-4953-93a5-1beb6732d482-audit-log\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.883634 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.883588 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e64a6d2-f1e8-4953-93a5-1beb6732d482-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.883875 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.883853 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0e64a6d2-f1e8-4953-93a5-1beb6732d482-metrics-server-audit-profiles\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.885762 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.885743 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-client-ca-bundle\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.885945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.885924 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-secret-metrics-server-client-certs\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.886014 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.885923 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0e64a6d2-f1e8-4953-93a5-1beb6732d482-secret-metrics-server-tls\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.890194 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.890168 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxw84\" (UniqueName: \"kubernetes.io/projected/0e64a6d2-f1e8-4953-93a5-1beb6732d482-kube-api-access-xxw84\") pod \"metrics-server-56668596dd-4xm4j\" (UID: \"0e64a6d2-f1e8-4953-93a5-1beb6732d482\") " pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:27.895813 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:27.895793 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:28.013675 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.013615 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:28.101407 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.101375 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87"] Jul 22 04:29:28.105550 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.105524 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.107673 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.107650 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-hxcrq\"" Jul 22 04:29:28.107795 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.107693 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 22 04:29:28.112282 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.112231 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87"] Jul 22 04:29:28.185409 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.185372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e838cdb-e77a-485f-b81c-093068166367-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.185568 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.185544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q5w2\" (UniqueName: \"kubernetes.io/projected/3e838cdb-e77a-485f-b81c-093068166367-kube-api-access-6q5w2\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.286864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.286825 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q5w2\" (UniqueName: \"kubernetes.io/projected/3e838cdb-e77a-485f-b81c-093068166367-kube-api-access-6q5w2\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.287030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.286897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e838cdb-e77a-485f-b81c-093068166367-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.287131 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:28.287041 2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 22 04:29:28.287131 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:28.287125 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e838cdb-e77a-485f-b81c-093068166367-monitoring-plugin-cert podName:3e838cdb-e77a-485f-b81c-093068166367 nodeName:}" failed. No retries permitted until 2026-07-22 04:29:28.787108707 +0000 UTC m=+292.876387860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3e838cdb-e77a-485f-b81c-093068166367-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-xnv87" (UID: "3e838cdb-e77a-485f-b81c-093068166367") : secret "monitoring-plugin-cert" not found Jul 22 04:29:28.295182 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.295155 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q5w2\" (UniqueName: \"kubernetes.io/projected/3e838cdb-e77a-485f-b81c-093068166367-kube-api-access-6q5w2\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.791387 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.791351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e838cdb-e77a-485f-b81c-093068166367-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:28.793947 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:28.793918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3e838cdb-e77a-485f-b81c-093068166367-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-xnv87\" (UID: \"3e838cdb-e77a-485f-b81c-093068166367\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:29.018521 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:29.018485 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:30.926171 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:29:30.926119 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4fnp2" podUID="256b8fe7-db10-4e43-9d5c-d75acc517fb2" Jul 22 04:29:32.014172 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:32.014139 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65a58f4e_fd6d_45c0_8d15_dcbe42d877d9.slice/crio-e1939ea280a1942155d55a5dfd02e6804f7af77bb10a6224ae30b379f40fde75 WatchSource:0}: Error finding container e1939ea280a1942155d55a5dfd02e6804f7af77bb10a6224ae30b379f40fde75: Status 404 returned error can't find the container with id e1939ea280a1942155d55a5dfd02e6804f7af77bb10a6224ae30b379f40fde75 Jul 22 04:29:32.161822 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.161796 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cdd7b794b-6qpfs"] Jul 22 04:29:32.277115 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:32.277019 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4926bdc_37e4_45c0_a27d_7a27efc8b53e.slice/crio-8437c2d89a77bb6ca270c8d7cd52dd987902296675c405f6353d6b44c3a30377 WatchSource:0}: Error finding container 8437c2d89a77bb6ca270c8d7cd52dd987902296675c405f6353d6b44c3a30377: Status 404 returned error can't find the container with id 8437c2d89a77bb6ca270c8d7cd52dd987902296675c405f6353d6b44c3a30377 Jul 22 04:29:32.386093 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.386049 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87"] Jul 22 04:29:32.390044 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.389978 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-56668596dd-4xm4j"] Jul 22 04:29:32.392084 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.391509 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-8674fd69d-5t2vd"] Jul 22 04:29:32.393681 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:32.393150 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e838cdb_e77a_485f_b81c_093068166367.slice/crio-3cb7d6cca8e33f7a42479b8c2f1ab4e6dc94196d86b785052c2afd38cbc181ef WatchSource:0}: Error finding container 3cb7d6cca8e33f7a42479b8c2f1ab4e6dc94196d86b785052c2afd38cbc181ef: Status 404 returned error can't find the container with id 3cb7d6cca8e33f7a42479b8c2f1ab4e6dc94196d86b785052c2afd38cbc181ef Jul 22 04:29:32.394660 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:32.394631 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7457d92_17c3_4353_9cd9_a88214127be4.slice/crio-a271bbc2a08f87dc16d2dae7e17f547b9f54c703a25aa3fb3dc1250f6ea8c66e WatchSource:0}: Error finding container a271bbc2a08f87dc16d2dae7e17f547b9f54c703a25aa3fb3dc1250f6ea8c66e: Status 404 returned error can't find the container with id a271bbc2a08f87dc16d2dae7e17f547b9f54c703a25aa3fb3dc1250f6ea8c66e Jul 22 04:29:32.397883 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:29:32.397826 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e64a6d2_f1e8_4953_93a5_1beb6732d482.slice/crio-f7a90958174f19acdeed2e8539e900c5e1e482a17bcf0363b0b84046dcd42e22 WatchSource:0}: Error finding container f7a90958174f19acdeed2e8539e900c5e1e482a17bcf0363b0b84046dcd42e22: Status 404 returned error can't find the container with id f7a90958174f19acdeed2e8539e900c5e1e482a17bcf0363b0b84046dcd42e22 Jul 22 04:29:32.831503 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.831455 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd7b794b-6qpfs" event={"ID":"c4926bdc-37e4-45c0-a27d-7a27efc8b53e","Type":"ContainerStarted","Data":"8437c2d89a77bb6ca270c8d7cd52dd987902296675c405f6353d6b44c3a30377"} Jul 22 04:29:32.833809 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.833711 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"a271bbc2a08f87dc16d2dae7e17f547b9f54c703a25aa3fb3dc1250f6ea8c66e"} Jul 22 04:29:32.835737 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.835677 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" event={"ID":"0e64a6d2-f1e8-4953-93a5-1beb6732d482","Type":"ContainerStarted","Data":"f7a90958174f19acdeed2e8539e900c5e1e482a17bcf0363b0b84046dcd42e22"} Jul 22 04:29:32.837386 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.837329 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" event={"ID":"3e838cdb-e77a-485f-b81c-093068166367","Type":"ContainerStarted","Data":"3cb7d6cca8e33f7a42479b8c2f1ab4e6dc94196d86b785052c2afd38cbc181ef"} Jul 22 04:29:32.849936 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.847371 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-nkzrk" event={"ID":"ba7434a6-9f90-47d6-9451-261245769db1","Type":"ContainerStarted","Data":"b7198638dedb33a0b67b4cdf1d8e03d3f31aef77a8f4b698734cbc7b58f4b7aa"} Jul 22 04:29:32.850169 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.849883 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:32.852971 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.852948 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7m876" event={"ID":"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9","Type":"ContainerStarted","Data":"e1939ea280a1942155d55a5dfd02e6804f7af77bb10a6224ae30b379f40fde75"} Jul 22 04:29:32.857599 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.857580 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-nkzrk" Jul 22 04:29:32.886493 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:32.885437 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-nkzrk" podStartSLOduration=1.61925162 podStartE2EDuration="18.885414198s" totalImagesPullingTime="17.266162578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:32.867313602 +0000 UTC m=+296.956592775" watchObservedRunningTime="2026-07-22 04:29:32.885414198 +0000 UTC m=+296.974693371" Jul 22 04:29:34.800101 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:34.799285 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4c7tk" Jul 22 04:29:37.060634 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:37.060439 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76fb9bc456-vqjgc"] Jul 22 04:29:37.316031 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:37.066688 2566 patch_prober.go:28] interesting pod/image-registry-76fb9bc456-vqjgc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 22 04:29:37.316031 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:37.066754 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:29:38.982934 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:38.982909 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 04:29:39.445580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.444603 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:29:39.890884 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.890792 2566 generic.go:356] "Generic (PLEG): container finished" podID="65a58f4e-fd6d-45c0-8d15-dcbe42d877d9" containerID="14b28462586bae7d846770cde6c59098b40619320ac7fe38dd42d9f2a5b30a7e" exitCode=0 Jul 22 04:29:39.891147 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.890885 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7m876" event={"ID":"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9","Type":"ContainerDied","Data":"14b28462586bae7d846770cde6c59098b40619320ac7fe38dd42d9f2a5b30a7e"} Jul 22 04:29:39.893499 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.893468 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" event={"ID":"26688b8c-fde4-486e-8527-cf9795f8355b","Type":"ContainerStarted","Data":"fe1b309a16788ffb2fe08407cfa07ffcc9c70073fec50341911fd851234f211b"} Jul 22 04:29:39.893616 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.893532 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" event={"ID":"26688b8c-fde4-486e-8527-cf9795f8355b","Type":"ContainerStarted","Data":"99b2dc53b8a0b4b6025b44de0c7694173040aeae2492cdbaddc6af6e63edfa30"} Jul 22 04:29:39.893616 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.893566 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" event={"ID":"26688b8c-fde4-486e-8527-cf9795f8355b","Type":"ContainerStarted","Data":"3cb1d5554d286ad32e545050efc836c5eb151c7f4b87b1a49e50d4fa4811f037"} Jul 22 04:29:39.895425 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.895375 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd7b794b-6qpfs" event={"ID":"c4926bdc-37e4-45c0-a27d-7a27efc8b53e","Type":"ContainerStarted","Data":"dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d"} Jul 22 04:29:39.898234 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.898210 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"56e2e21bbc87a87e8bef7e10035d0818a18e992c4aa7ac47bc669ac9fa26f192"} Jul 22 04:29:39.898352 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.898274 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"f885dc75df7f39f0ffe065791a113a671495e5656cf16363d24fc9cd49bc9f27"} Jul 22 04:29:39.898352 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.898315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"28e1bc1e355e2bc5915ca4820aecbe2154bcaa6e254232c588881779e07c890f"} Jul 22 04:29:39.899902 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.899865 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" event={"ID":"0e64a6d2-f1e8-4953-93a5-1beb6732d482","Type":"ContainerStarted","Data":"721bbc011a04636a8e7a5c3439150d833f0b5fc0a8d73996c98932345050a842"} Jul 22 04:29:39.902092 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.902052 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" event={"ID":"65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931","Type":"ContainerStarted","Data":"20c26afe576806701dbbdf7d5aaaa5f60500986168859dce60130b66707858c6"} Jul 22 04:29:39.903579 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.903550 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" event={"ID":"3e838cdb-e77a-485f-b81c-093068166367","Type":"ContainerStarted","Data":"8791007d5aa6930f371e8b5d502fdefb930cf7ab459eb876795539a63f97c494"} Jul 22 04:29:39.928016 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.927945 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-z7ln7" podStartSLOduration=2.397209912 podStartE2EDuration="16.927922801s" totalImagesPullingTime="14.530712889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:39.92588838 +0000 UTC m=+304.015167598" watchObservedRunningTime="2026-07-22 04:29:39.927922801 +0000 UTC m=+304.017201976" Jul 22 04:29:39.946054 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.946003 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cdd7b794b-6qpfs" podStartSLOduration=6.300809082 podStartE2EDuration="12.945980807s" totalImagesPullingTime="6.645171725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:39.945600229 +0000 UTC m=+304.034879403" watchObservedRunningTime="2026-07-22 04:29:39.945980807 +0000 UTC m=+304.035259980" Jul 22 04:29:39.966677 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.965746 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" podStartSLOduration=5.4200702419999995 podStartE2EDuration="11.965726572s" totalImagesPullingTime="6.54565633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:39.963460138 +0000 UTC m=+304.052739311" watchObservedRunningTime="2026-07-22 04:29:39.965726572 +0000 UTC m=+304.055005746" Jul 22 04:29:39.988681 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:39.988624 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-qlfrg" podStartSLOduration=3.107306352 podStartE2EDuration="16.988605945s" totalImagesPullingTime="13.881299593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:39.988348024 +0000 UTC m=+304.077627404" watchObservedRunningTime="2026-07-22 04:29:39.988605945 +0000 UTC m=+304.077885118" Jul 22 04:29:40.023118 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:40.023048 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" podStartSLOduration=6.476448835 podStartE2EDuration="13.023025762s" totalImagesPullingTime="6.546576927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:40.021860612 +0000 UTC m=+304.111139786" watchObservedRunningTime="2026-07-22 04:29:40.023025762 +0000 UTC m=+304.112304935" Jul 22 04:29:40.912550 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:40.912512 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7m876" event={"ID":"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9","Type":"ContainerStarted","Data":"f7dafe93b2e0bd4e0627e6bb034fcf02013d837a7e49400bfd710f32f1306874"} Jul 22 04:29:40.912714 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:40.912611 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7m876" event={"ID":"65a58f4e-fd6d-45c0-8d15-dcbe42d877d9","Type":"ContainerStarted","Data":"ba68b5ceeca10995d48cc91e63c5ce1da83e1f380a7739fca1dc260fdc843527"} Jul 22 04:29:40.913611 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:40.913586 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:40.918902 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:40.918878 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-xnv87" Jul 22 04:29:40.932723 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:40.932671 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7m876" podStartSLOduration=10.893308511 podStartE2EDuration="17.932651669s" totalImagesPullingTime="6.843649414s" totalInitContainerRuntime="195.693744ms" isStatefulPod=true podCreationTimestamp="2026-07-22 04:29:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:40.93097662 +0000 UTC m=+305.020255846" watchObservedRunningTime="2026-07-22 04:29:40.932651669 +0000 UTC m=+305.021930842" Jul 22 04:29:41.918113 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:41.918054 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"d27ceca2bf28881cf46857ba3bb14355e5ee59adfbb7eef46de630247a93b9f4"} Jul 22 04:29:41.918614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:41.918140 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"31b40f438e6fee1491a40833c3b90824d116589e5215ce26cc61f53eb50f5a4a"} Jul 22 04:29:41.918614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:41.918167 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" event={"ID":"d7457d92-17c3-4353-9cd9-a88214127be4","Type":"ContainerStarted","Data":"509ca97c3a8465a7338ffe9c05becb69ba676e936fb66fcc9e4532935fe7a485"} Jul 22 04:29:41.942593 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:41.942544 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" podStartSLOduration=8.064554793 podStartE2EDuration="15.942527462s" totalImagesPullingTime="7.877972669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:29:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:29:41.941888268 +0000 UTC m=+306.031167482" watchObservedRunningTime="2026-07-22 04:29:41.942527462 +0000 UTC m=+306.031806634" Jul 22 04:29:42.922562 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:42.922528 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:29:42.925852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:42.925838 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:29:45.004924 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:45.004890 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdd7b794b-6qpfs"] Jul 22 04:29:47.065695 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:47.065664 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:29:47.896390 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:47.896352 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:29:48.013961 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:48.013932 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:48.014141 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:48.013977 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:29:48.936246 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:29:48.936219 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-8674fd69d-5t2vd" Jul 22 04:30:02.089824 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.089783 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" containerName="registry" containerID="cri-o://88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e" gracePeriod=30 Jul 22 04:30:02.360874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.360804 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:30:02.406487 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406457 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-installation-pull-secrets\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406487 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406492 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-certificates\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406519 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2k6d\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-kube-api-access-q2k6d\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406537 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-trusted-ca\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406664 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-bound-sa-token\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406835 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406715 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-image-registry-private-configuration\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406835 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406769 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406835 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-ca-trust-extracted\") pod \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\" (UID: \"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c\") " Jul 22 04:30:02.406999 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406970 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:30:02.407054 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.406998 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:30:02.407156 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.407140 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-certificates\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.407298 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.407266 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-trusted-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.409209 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.409183 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:30:02.409318 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.409229 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:30:02.409318 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.409231 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:30:02.409318 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.409273 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-kube-api-access-q2k6d" (OuterVolumeSpecName: "kube-api-access-q2k6d") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "kube-api-access-q2k6d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:30:02.409533 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.409514 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:30:02.415829 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.415805 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" (UID: "a1c5bcfb-34cc-42dc-9479-f0a116b58a1c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:30:02.507854 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.507815 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2k6d\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-kube-api-access-q2k6d\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.507854 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.507847 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-bound-sa-token\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.507854 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.507857 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-image-registry-private-configuration\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.507854 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.507866 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-registry-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.508154 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.507875 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-ca-trust-extracted\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:02.508154 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:02.507883 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c-installation-pull-secrets\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:03.005331 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.005301 2566 generic.go:356] "Generic (PLEG): container finished" podID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" containerID="88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e" exitCode=0 Jul 22 04:30:03.005485 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.005341 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" event={"ID":"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c","Type":"ContainerDied","Data":"88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e"} Jul 22 04:30:03.005485 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.005348 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" Jul 22 04:30:03.005485 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.005375 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76fb9bc456-vqjgc" event={"ID":"a1c5bcfb-34cc-42dc-9479-f0a116b58a1c","Type":"ContainerDied","Data":"d1b2a2210aa97fb7dc92cf817aeebaa1ae4bce54ca42a9af922c940ed936190a"} Jul 22 04:30:03.005485 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.005406 2566 scope.go:117] "RemoveContainer" containerID="88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e" Jul 22 04:30:03.013987 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.013972 2566 scope.go:117] "RemoveContainer" containerID="88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e" Jul 22 04:30:03.014265 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:30:03.014242 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e\": container with ID starting with 88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e not found: ID does not exist" containerID="88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e" Jul 22 04:30:03.014338 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.014272 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e"} err="failed to get container status \"88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e\": rpc error: code = NotFound desc = could not find container \"88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e\": container with ID starting with 88fb9c13469f976a9706e2240d475d453c3a8d7ca56f9699d9bedf0e4d5a340e not found: ID does not exist" Jul 22 04:30:03.022095 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.022059 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76fb9bc456-vqjgc"] Jul 22 04:30:03.028418 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:03.028292 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76fb9bc456-vqjgc"] Jul 22 04:30:04.929850 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:04.929815 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" path="/var/lib/kubelet/pods/a1c5bcfb-34cc-42dc-9479-f0a116b58a1c/volumes" Jul 22 04:30:08.019341 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:08.019312 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:30:08.023451 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:08.023431 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-56668596dd-4xm4j" Jul 22 04:30:10.025106 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.025035 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cdd7b794b-6qpfs" podUID="c4926bdc-37e4-45c0-a27d-7a27efc8b53e" containerName="console" containerID="cri-o://dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d" gracePeriod=15 Jul 22 04:30:10.291663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.291596 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:30:10.377814 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377782 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-config\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.377814 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377816 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m4zp\" (UniqueName: \"kubernetes.io/projected/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-kube-api-access-5m4zp\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.378017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377852 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-serving-cert\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.378017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377874 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-service-ca\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.378017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377901 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-trusted-ca-bundle\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.378017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377941 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-oauth-serving-cert\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.378017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.377972 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-oauth-config\") pod \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\" (UID: \"c4926bdc-37e4-45c0-a27d-7a27efc8b53e\") " Jul 22 04:30:10.378292 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.378259 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-config" (OuterVolumeSpecName: "console-config") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:30:10.378422 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.378393 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-service-ca" (OuterVolumeSpecName: "service-ca") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:30:10.378481 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.378431 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:30:10.378694 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.378667 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:30:10.380118 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.380094 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:30:10.380209 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.380146 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-kube-api-access-5m4zp" (OuterVolumeSpecName: "kube-api-access-5m4zp") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "kube-api-access-5m4zp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:30:10.380209 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.380183 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c4926bdc-37e4-45c0-a27d-7a27efc8b53e" (UID: "c4926bdc-37e4-45c0-a27d-7a27efc8b53e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:30:10.479511 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479486 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-serving-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:10.479511 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479509 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:10.479667 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479518 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-trusted-ca-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:10.479667 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479528 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-oauth-serving-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:10.479667 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479536 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-oauth-config\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:10.479667 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479544 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-console-config\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:10.479667 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:10.479552 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m4zp\" (UniqueName: \"kubernetes.io/projected/c4926bdc-37e4-45c0-a27d-7a27efc8b53e-kube-api-access-5m4zp\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:30:11.038372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.038339 2566 generic.go:356] "Generic (PLEG): container finished" podID="c4926bdc-37e4-45c0-a27d-7a27efc8b53e" containerID="dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d" exitCode=0 Jul 22 04:30:11.038785 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.038384 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cdd7b794b-6qpfs" Jul 22 04:30:11.038785 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.038395 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd7b794b-6qpfs" event={"ID":"c4926bdc-37e4-45c0-a27d-7a27efc8b53e","Type":"ContainerDied","Data":"dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d"} Jul 22 04:30:11.038785 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.038448 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cdd7b794b-6qpfs" event={"ID":"c4926bdc-37e4-45c0-a27d-7a27efc8b53e","Type":"ContainerDied","Data":"8437c2d89a77bb6ca270c8d7cd52dd987902296675c405f6353d6b44c3a30377"} Jul 22 04:30:11.038785 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.038469 2566 scope.go:117] "RemoveContainer" containerID="dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d" Jul 22 04:30:11.052522 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.052447 2566 scope.go:117] "RemoveContainer" containerID="dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d" Jul 22 04:30:11.052776 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:30:11.052755 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d\": container with ID starting with dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d not found: ID does not exist" containerID="dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d" Jul 22 04:30:11.052846 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.052783 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d"} err="failed to get container status \"dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d\": rpc error: code = NotFound desc = could not find container \"dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d\": container with ID starting with dda92c342bda397cc8442b428aef0942d143d8308a901f6cdd172e66e7fef48d not found: ID does not exist" Jul 22 04:30:11.053771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.053754 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cdd7b794b-6qpfs"] Jul 22 04:30:11.057384 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:11.057365 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cdd7b794b-6qpfs"] Jul 22 04:30:12.930014 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:12.929979 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4926bdc-37e4-45c0-a27d-7a27efc8b53e" path="/var/lib/kubelet/pods/c4926bdc-37e4-45c0-a27d-7a27efc8b53e/volumes" Jul 22 04:30:19.074252 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:19.074216 2566 generic.go:356] "Generic (PLEG): container finished" podID="5d024499-091d-499f-8f5c-3673b0a5b833" containerID="43725464410d68a336ae9561ad98258280ad0bd2179559b51bae1b6537ee0052" exitCode=0 Jul 22 04:30:19.074717 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:19.074292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" event={"ID":"5d024499-091d-499f-8f5c-3673b0a5b833","Type":"ContainerDied","Data":"43725464410d68a336ae9561ad98258280ad0bd2179559b51bae1b6537ee0052"} Jul 22 04:30:19.074717 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:19.074576 2566 scope.go:117] "RemoveContainer" containerID="43725464410d68a336ae9561ad98258280ad0bd2179559b51bae1b6537ee0052" Jul 22 04:30:20.079789 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:20.079752 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-thb4s" event={"ID":"5d024499-091d-499f-8f5c-3673b0a5b833","Type":"ContainerStarted","Data":"3d38b7fe90fcdbc2a75f73f5c052e0cd4cc395ef7a5d3c78d4ace38b4a40a41d"} Jul 22 04:30:50.800763 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:50.800722 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:30:50.803030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:50.803011 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/256b8fe7-db10-4e43-9d5c-d75acc517fb2-metrics-certs\") pod \"network-metrics-daemon-4fnp2\" (UID: \"256b8fe7-db10-4e43-9d5c-d75acc517fb2\") " pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:30:51.028673 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.028645 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s9hfn\"" Jul 22 04:30:51.037432 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.037413 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4fnp2" Jul 22 04:30:51.132823 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.132792 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76cdff6674-6sdr9"] Jul 22 04:30:51.133295 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.133279 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" containerName="registry" Jul 22 04:30:51.133341 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.133299 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" containerName="registry" Jul 22 04:30:51.133341 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.133321 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4926bdc-37e4-45c0-a27d-7a27efc8b53e" containerName="console" Jul 22 04:30:51.133341 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.133330 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4926bdc-37e4-45c0-a27d-7a27efc8b53e" containerName="console" Jul 22 04:30:51.133434 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.133404 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4926bdc-37e4-45c0-a27d-7a27efc8b53e" containerName="console" Jul 22 04:30:51.133434 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.133417 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1c5bcfb-34cc-42dc-9479-f0a116b58a1c" containerName="registry" Jul 22 04:30:51.136255 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.136235 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.139170 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.138596 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 04:30:51.139170 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.138852 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 04:30:51.139170 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.139157 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 04:30:51.139549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.139320 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 04:30:51.139549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.139358 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 04:30:51.140036 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.140013 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-r5m4w\"" Jul 22 04:30:51.143574 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.143527 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 04:30:51.147182 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.147160 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cdff6674-6sdr9"] Jul 22 04:30:51.172332 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.172311 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4fnp2"] Jul 22 04:30:51.175625 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:30:51.175604 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod256b8fe7_db10_4e43_9d5c_d75acc517fb2.slice/crio-bb30207639dd3f8431829636f9a7f600376fa8aa36ec1c6d659ec67e38bfe9e1 WatchSource:0}: Error finding container bb30207639dd3f8431829636f9a7f600376fa8aa36ec1c6d659ec67e38bfe9e1: Status 404 returned error can't find the container with id bb30207639dd3f8431829636f9a7f600376fa8aa36ec1c6d659ec67e38bfe9e1 Jul 22 04:30:51.203706 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203678 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-oauth-config\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.203706 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203710 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-service-ca\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.203929 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203726 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-oauth-serving-cert\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.203929 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203758 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-config\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.203929 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-serving-cert\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.203929 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kq29\" (UniqueName: \"kubernetes.io/projected/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-kube-api-access-4kq29\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.204095 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.203934 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-trusted-ca-bundle\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.204401 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.204380 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4fnp2" event={"ID":"256b8fe7-db10-4e43-9d5c-d75acc517fb2","Type":"ContainerStarted","Data":"bb30207639dd3f8431829636f9a7f600376fa8aa36ec1c6d659ec67e38bfe9e1"} Jul 22 04:30:51.304657 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.304623 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-trusted-ca-bundle\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.304823 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.304664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-oauth-config\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.304823 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.304680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-service-ca\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.304954 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.304877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-oauth-serving-cert\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.304954 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.304924 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-config\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.305485 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-serving-cert\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.305580 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kq29\" (UniqueName: \"kubernetes.io/projected/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-kube-api-access-4kq29\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.305865 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-service-ca\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.306014 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-config\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.306123 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-oauth-serving-cert\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.306310 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-trusted-ca-bundle\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.311057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.308682 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-serving-cert\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.312461 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.312428 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-oauth-config\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.313795 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.313774 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kq29\" (UniqueName: \"kubernetes.io/projected/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-kube-api-access-4kq29\") pod \"console-76cdff6674-6sdr9\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.449195 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.449161 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:30:51.572855 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:51.572831 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cdff6674-6sdr9"] Jul 22 04:30:51.575159 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:30:51.575121 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb45fae8d_1c16_4bef_ac5c_fea95b8bef80.slice/crio-02ee9f3d0e875ef84be6ec5863b492f9cf4da8c00229c2fd2d44175135a9323d WatchSource:0}: Error finding container 02ee9f3d0e875ef84be6ec5863b492f9cf4da8c00229c2fd2d44175135a9323d: Status 404 returned error can't find the container with id 02ee9f3d0e875ef84be6ec5863b492f9cf4da8c00229c2fd2d44175135a9323d Jul 22 04:30:52.209575 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:52.209539 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cdff6674-6sdr9" event={"ID":"b45fae8d-1c16-4bef-ac5c-fea95b8bef80","Type":"ContainerStarted","Data":"ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180"} Jul 22 04:30:52.209972 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:52.209590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cdff6674-6sdr9" event={"ID":"b45fae8d-1c16-4bef-ac5c-fea95b8bef80","Type":"ContainerStarted","Data":"02ee9f3d0e875ef84be6ec5863b492f9cf4da8c00229c2fd2d44175135a9323d"} Jul 22 04:30:52.228755 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:52.228708 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76cdff6674-6sdr9" podStartSLOduration=1.228692121 podStartE2EDuration="1.228692121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:30:52.227339027 +0000 UTC m=+376.316618216" watchObservedRunningTime="2026-07-22 04:30:52.228692121 +0000 UTC m=+376.317971301" Jul 22 04:30:53.214892 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:53.214853 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4fnp2" event={"ID":"256b8fe7-db10-4e43-9d5c-d75acc517fb2","Type":"ContainerStarted","Data":"4fc0521e4d0112242f34d2566f3ce6ecb183c172efae7bc6c3427d87a8db558b"} Jul 22 04:30:53.215413 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:53.214910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4fnp2" event={"ID":"256b8fe7-db10-4e43-9d5c-d75acc517fb2","Type":"ContainerStarted","Data":"b1b4058675f0db0a16947e2159b77e2de1dbf28515faa9c17fdf4a8caea60eec"} Jul 22 04:30:53.231713 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:30:53.231658 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4fnp2" podStartSLOduration=374.836395352 podStartE2EDuration="6m16.231642331s" totalImagesPullingTime="1.395246979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:24:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:30:53.229639229 +0000 UTC m=+377.318918423" watchObservedRunningTime="2026-07-22 04:30:53.231642331 +0000 UTC m=+377.320921502" Jul 22 04:31:01.449751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:31:01.449708 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:31:01.450257 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:31:01.449792 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:31:01.454400 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:31:01.454377 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:31:02.254262 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:31:02.254224 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:32:07.691913 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.691825 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56d7d79d55-gwgzb"] Jul 22 04:32:07.694764 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.694743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.706470 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.706446 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d7d79d55-gwgzb"] Jul 22 04:32:07.802138 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-console-config\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.802138 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802140 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-trusted-ca-bundle\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.802344 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802170 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-oauth-config\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.802344 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802256 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-service-ca\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.802344 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-oauth-serving-cert\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.802344 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802293 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-serving-cert\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.802470 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.802353 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glpnt\" (UniqueName: \"kubernetes.io/projected/1c513081-93dd-4c37-a8bd-797233855254-kube-api-access-glpnt\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903385 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-oauth-config\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903385 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903399 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-service-ca\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903618 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903420 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-oauth-serving-cert\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903618 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-serving-cert\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903618 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903476 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glpnt\" (UniqueName: \"kubernetes.io/projected/1c513081-93dd-4c37-a8bd-797233855254-kube-api-access-glpnt\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903618 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-console-config\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.903618 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.903513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-trusted-ca-bundle\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.904220 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.904193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-service-ca\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.904336 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.904228 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-oauth-serving-cert\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.904336 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.904273 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-console-config\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.904469 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.904446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-trusted-ca-bundle\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.905923 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.905891 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-oauth-config\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.906064 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.906043 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-serving-cert\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:07.913132 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:07.913051 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glpnt\" (UniqueName: \"kubernetes.io/projected/1c513081-93dd-4c37-a8bd-797233855254-kube-api-access-glpnt\") pod \"console-56d7d79d55-gwgzb\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:08.004263 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:08.004167 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:08.162961 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:08.162927 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d7d79d55-gwgzb"] Jul 22 04:32:08.166754 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:32:08.166728 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c513081_93dd_4c37_a8bd_797233855254.slice/crio-8f02f4c31cdc0d2212de48541899ced07afda9447aa98340cfb3cac0c9460433 WatchSource:0}: Error finding container 8f02f4c31cdc0d2212de48541899ced07afda9447aa98340cfb3cac0c9460433: Status 404 returned error can't find the container with id 8f02f4c31cdc0d2212de48541899ced07afda9447aa98340cfb3cac0c9460433 Jul 22 04:32:08.506432 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:08.506396 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d7d79d55-gwgzb" event={"ID":"1c513081-93dd-4c37-a8bd-797233855254","Type":"ContainerStarted","Data":"a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82"} Jul 22 04:32:08.506639 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:08.506444 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d7d79d55-gwgzb" event={"ID":"1c513081-93dd-4c37-a8bd-797233855254","Type":"ContainerStarted","Data":"8f02f4c31cdc0d2212de48541899ced07afda9447aa98340cfb3cac0c9460433"} Jul 22 04:32:08.523842 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:08.523795 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56d7d79d55-gwgzb" podStartSLOduration=1.52377963 podStartE2EDuration="1.52377963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:32:08.522463992 +0000 UTC m=+452.611743164" watchObservedRunningTime="2026-07-22 04:32:08.52377963 +0000 UTC m=+452.613058801" Jul 22 04:32:18.005331 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:18.005293 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:18.005751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:18.005373 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:18.010380 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:18.010356 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:18.551399 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:18.551371 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:32:18.598713 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:18.598664 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cdff6674-6sdr9"] Jul 22 04:32:43.619384 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:43.619345 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76cdff6674-6sdr9" podUID="b45fae8d-1c16-4bef-ac5c-fea95b8bef80" containerName="console" containerID="cri-o://ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180" gracePeriod=15 Jul 22 04:32:43.861612 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:43.861590 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:32:44.009653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009618 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-service-ca\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.009653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009657 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-serving-cert\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.009909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009700 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-config\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.009909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009719 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-trusted-ca-bundle\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.009909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009745 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-oauth-serving-cert\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.009909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009774 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-oauth-config\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.009909 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.009887 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4kq29\" (UniqueName: \"kubernetes.io/projected/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-kube-api-access-4kq29\") pod \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\" (UID: \"b45fae8d-1c16-4bef-ac5c-fea95b8bef80\") " Jul 22 04:32:44.010200 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.010043 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-service-ca" (OuterVolumeSpecName: "service-ca") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:32:44.010200 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.010195 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.010303 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.010125 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-config" (OuterVolumeSpecName: "console-config") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:32:44.010339 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.010323 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:32:44.010400 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.010340 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:32:44.012114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.012091 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:32:44.012247 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.012163 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-kube-api-access-4kq29" (OuterVolumeSpecName: "kube-api-access-4kq29") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "kube-api-access-4kq29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:32:44.012310 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.012253 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b45fae8d-1c16-4bef-ac5c-fea95b8bef80" (UID: "b45fae8d-1c16-4bef-ac5c-fea95b8bef80"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:32:44.111469 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.111435 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kq29\" (UniqueName: \"kubernetes.io/projected/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-kube-api-access-4kq29\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.111469 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.111464 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-serving-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.111663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.111477 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-config\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.111663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.111491 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-trusted-ca-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.111663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.111503 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-oauth-serving-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.111663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.111515 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b45fae8d-1c16-4bef-ac5c-fea95b8bef80-console-oauth-config\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:32:44.651114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.651059 2566 generic.go:356] "Generic (PLEG): container finished" podID="b45fae8d-1c16-4bef-ac5c-fea95b8bef80" containerID="ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180" exitCode=0 Jul 22 04:32:44.651114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.651098 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cdff6674-6sdr9" event={"ID":"b45fae8d-1c16-4bef-ac5c-fea95b8bef80","Type":"ContainerDied","Data":"ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180"} Jul 22 04:32:44.651114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.651117 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cdff6674-6sdr9" Jul 22 04:32:44.651571 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.651152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cdff6674-6sdr9" event={"ID":"b45fae8d-1c16-4bef-ac5c-fea95b8bef80","Type":"ContainerDied","Data":"02ee9f3d0e875ef84be6ec5863b492f9cf4da8c00229c2fd2d44175135a9323d"} Jul 22 04:32:44.651571 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.651174 2566 scope.go:117] "RemoveContainer" containerID="ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180" Jul 22 04:32:44.659484 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.659465 2566 scope.go:117] "RemoveContainer" containerID="ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180" Jul 22 04:32:44.659755 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:32:44.659738 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180\": container with ID starting with ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180 not found: ID does not exist" containerID="ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180" Jul 22 04:32:44.659800 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.659765 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180"} err="failed to get container status \"ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180\": rpc error: code = NotFound desc = could not find container \"ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180\": container with ID starting with ced7aa5dfbf8c4c69f4a63acae4d4f595f7080fcde5056e04cbbfd0b8354c180 not found: ID does not exist" Jul 22 04:32:44.670919 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.670865 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cdff6674-6sdr9"] Jul 22 04:32:44.672730 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.672711 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76cdff6674-6sdr9"] Jul 22 04:32:44.930496 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:44.930423 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b45fae8d-1c16-4bef-ac5c-fea95b8bef80" path="/var/lib/kubelet/pods/b45fae8d-1c16-4bef-ac5c-fea95b8bef80/volumes" Jul 22 04:32:54.877120 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.877086 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v"] Jul 22 04:32:54.877491 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.877400 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b45fae8d-1c16-4bef-ac5c-fea95b8bef80" containerName="console" Jul 22 04:32:54.877491 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.877412 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b45fae8d-1c16-4bef-ac5c-fea95b8bef80" containerName="console" Jul 22 04:32:54.877491 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.877480 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b45fae8d-1c16-4bef-ac5c-fea95b8bef80" containerName="console" Jul 22 04:32:54.881847 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.881829 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:54.884290 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.884267 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lsdr8\"" Jul 22 04:32:54.884409 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.884318 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 04:32:54.885030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.885014 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 04:32:54.889615 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.889592 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v"] Jul 22 04:32:54.901688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.901665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk799\" (UniqueName: \"kubernetes.io/projected/e267ab38-65f4-400f-855d-d84818fd9bc0-kube-api-access-sk799\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:54.901784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.901699 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:54.901784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:54.901718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.002120 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.002088 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk799\" (UniqueName: \"kubernetes.io/projected/e267ab38-65f4-400f-855d-d84818fd9bc0-kube-api-access-sk799\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.002266 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.002125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.002266 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.002144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.002456 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.002441 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.002491 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.002471 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.009983 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.009962 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk799\" (UniqueName: \"kubernetes.io/projected/e267ab38-65f4-400f-855d-d84818fd9bc0-kube-api-access-sk799\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.191615 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.191582 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:32:55.318403 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.318371 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v"] Jul 22 04:32:55.321614 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:32:55.321587 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode267ab38_65f4_400f_855d_d84818fd9bc0.slice/crio-a8908b043f283e6775285d766c625ff7b7bbc8b781f087a04c59434694b5c930 WatchSource:0}: Error finding container a8908b043f283e6775285d766c625ff7b7bbc8b781f087a04c59434694b5c930: Status 404 returned error can't find the container with id a8908b043f283e6775285d766c625ff7b7bbc8b781f087a04c59434694b5c930 Jul 22 04:32:55.696108 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:32:55.696058 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" event={"ID":"e267ab38-65f4-400f-855d-d84818fd9bc0","Type":"ContainerStarted","Data":"a8908b043f283e6775285d766c625ff7b7bbc8b781f087a04c59434694b5c930"} Jul 22 04:33:02.734527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:02.734493 2566 generic.go:356] "Generic (PLEG): container finished" podID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerID="a7ec2941f2d713e9527b3b940218bc5f1ed9155e8ba8d3fdd3821a06c783a289" exitCode=0 Jul 22 04:33:02.734927 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:02.734586 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" event={"ID":"e267ab38-65f4-400f-855d-d84818fd9bc0","Type":"ContainerDied","Data":"a7ec2941f2d713e9527b3b940218bc5f1ed9155e8ba8d3fdd3821a06c783a289"} Jul 22 04:33:04.745506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:04.745420 2566 generic.go:356] "Generic (PLEG): container finished" podID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerID="ac6f4ea8eb0ffef9008bec0144cd0c6fbb4a20da6eba7ee6239a53176eea18ee" exitCode=0 Jul 22 04:33:04.745506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:04.745472 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" event={"ID":"e267ab38-65f4-400f-855d-d84818fd9bc0","Type":"ContainerDied","Data":"ac6f4ea8eb0ffef9008bec0144cd0c6fbb4a20da6eba7ee6239a53176eea18ee"} Jul 22 04:33:11.782369 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:11.782332 2566 generic.go:356] "Generic (PLEG): container finished" podID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerID="453269ae0a4dd99a636e7029a9b9e59eff6abff6392eba5b5d2de7cbfc206525" exitCode=0 Jul 22 04:33:11.782750 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:11.782417 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" event={"ID":"e267ab38-65f4-400f-855d-d84818fd9bc0","Type":"ContainerDied","Data":"453269ae0a4dd99a636e7029a9b9e59eff6abff6392eba5b5d2de7cbfc206525"} Jul 22 04:33:12.916573 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:12.916544 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:33:13.059561 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.059462 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk799\" (UniqueName: \"kubernetes.io/projected/e267ab38-65f4-400f-855d-d84818fd9bc0-kube-api-access-sk799\") pod \"e267ab38-65f4-400f-855d-d84818fd9bc0\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " Jul 22 04:33:13.059561 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.059514 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-bundle\") pod \"e267ab38-65f4-400f-855d-d84818fd9bc0\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " Jul 22 04:33:13.059561 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.059558 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-util\") pod \"e267ab38-65f4-400f-855d-d84818fd9bc0\" (UID: \"e267ab38-65f4-400f-855d-d84818fd9bc0\") " Jul 22 04:33:13.060275 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.060243 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-bundle" (OuterVolumeSpecName: "bundle") pod "e267ab38-65f4-400f-855d-d84818fd9bc0" (UID: "e267ab38-65f4-400f-855d-d84818fd9bc0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:33:13.061721 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.061697 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e267ab38-65f4-400f-855d-d84818fd9bc0-kube-api-access-sk799" (OuterVolumeSpecName: "kube-api-access-sk799") pod "e267ab38-65f4-400f-855d-d84818fd9bc0" (UID: "e267ab38-65f4-400f-855d-d84818fd9bc0"). InnerVolumeSpecName "kube-api-access-sk799". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:33:13.063637 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.063614 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-util" (OuterVolumeSpecName: "util") pod "e267ab38-65f4-400f-855d-d84818fd9bc0" (UID: "e267ab38-65f4-400f-855d-d84818fd9bc0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:33:13.161062 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.161023 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:13.161062 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.161055 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e267ab38-65f4-400f-855d-d84818fd9bc0-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:13.161062 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.161083 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk799\" (UniqueName: \"kubernetes.io/projected/e267ab38-65f4-400f-855d-d84818fd9bc0-kube-api-access-sk799\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:13.794548 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.794516 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" event={"ID":"e267ab38-65f4-400f-855d-d84818fd9bc0","Type":"ContainerDied","Data":"a8908b043f283e6775285d766c625ff7b7bbc8b781f087a04c59434694b5c930"} Jul 22 04:33:13.794548 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.794544 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540pp94v" Jul 22 04:33:13.794773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:13.794564 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8908b043f283e6775285d766c625ff7b7bbc8b781f087a04c59434694b5c930" Jul 22 04:33:17.826816 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.826777 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz"] Jul 22 04:33:17.827417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827288 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="pull" Jul 22 04:33:17.827417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827308 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="pull" Jul 22 04:33:17.827417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827336 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="util" Jul 22 04:33:17.827417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827345 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="util" Jul 22 04:33:17.827417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827360 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="extract" Jul 22 04:33:17.827417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827369 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="extract" Jul 22 04:33:17.827782 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.827449 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e267ab38-65f4-400f-855d-d84818fd9bc0" containerName="extract" Jul 22 04:33:17.868390 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.868351 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz"] Jul 22 04:33:17.868539 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.868458 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:17.870719 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.870699 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 04:33:17.870719 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.870707 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:33:17.870993 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:17.870978 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-lhs82\"" Jul 22 04:33:18.004827 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.004775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e812655-3432-4bb3-bb38-25c515b8d91f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tqbdz\" (UID: \"2e812655-3432-4bb3-bb38-25c515b8d91f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.005034 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.004883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dffdt\" (UniqueName: \"kubernetes.io/projected/2e812655-3432-4bb3-bb38-25c515b8d91f-kube-api-access-dffdt\") pod \"cert-manager-operator-controller-manager-55c444bf86-tqbdz\" (UID: \"2e812655-3432-4bb3-bb38-25c515b8d91f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.105621 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.105536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dffdt\" (UniqueName: \"kubernetes.io/projected/2e812655-3432-4bb3-bb38-25c515b8d91f-kube-api-access-dffdt\") pod \"cert-manager-operator-controller-manager-55c444bf86-tqbdz\" (UID: \"2e812655-3432-4bb3-bb38-25c515b8d91f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.105771 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.105628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e812655-3432-4bb3-bb38-25c515b8d91f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tqbdz\" (UID: \"2e812655-3432-4bb3-bb38-25c515b8d91f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.106020 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.106001 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2e812655-3432-4bb3-bb38-25c515b8d91f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tqbdz\" (UID: \"2e812655-3432-4bb3-bb38-25c515b8d91f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.115254 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.115224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dffdt\" (UniqueName: \"kubernetes.io/projected/2e812655-3432-4bb3-bb38-25c515b8d91f-kube-api-access-dffdt\") pod \"cert-manager-operator-controller-manager-55c444bf86-tqbdz\" (UID: \"2e812655-3432-4bb3-bb38-25c515b8d91f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.177413 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.177368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" Jul 22 04:33:18.339666 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.339481 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz"] Jul 22 04:33:18.342948 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:33:18.342914 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e812655_3432_4bb3_bb38_25c515b8d91f.slice/crio-f226fea06dd2dee8f611f46e6d9385d7d669fed70554c9441fe0d5d6608d1298 WatchSource:0}: Error finding container f226fea06dd2dee8f611f46e6d9385d7d669fed70554c9441fe0d5d6608d1298: Status 404 returned error can't find the container with id f226fea06dd2dee8f611f46e6d9385d7d669fed70554c9441fe0d5d6608d1298 Jul 22 04:33:18.817639 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:18.817604 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" event={"ID":"2e812655-3432-4bb3-bb38-25c515b8d91f","Type":"ContainerStarted","Data":"f226fea06dd2dee8f611f46e6d9385d7d669fed70554c9441fe0d5d6608d1298"} Jul 22 04:33:20.828236 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:20.828202 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" event={"ID":"2e812655-3432-4bb3-bb38-25c515b8d91f","Type":"ContainerStarted","Data":"18f4015c309084b3d5947abe47c130e8310d44b001947cd87c64330ff82f182e"} Jul 22 04:33:20.848473 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:20.848362 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tqbdz" podStartSLOduration=1.612892042 podStartE2EDuration="3.848342398s" totalImagesPullingTime="2.235450356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:20.846497942 +0000 UTC m=+524.935777126" watchObservedRunningTime="2026-07-22 04:33:20.848342398 +0000 UTC m=+524.937621572" Jul 22 04:33:21.760593 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.760560 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tl5d2"] Jul 22 04:33:21.776899 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.776872 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tl5d2"] Jul 22 04:33:21.777045 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.776978 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:21.779005 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.778982 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 04:33:21.780189 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.780105 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 04:33:21.780189 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.780119 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-fd54v\"" Jul 22 04:33:21.781956 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.781932 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g"] Jul 22 04:33:21.796981 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.796959 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g"] Jul 22 04:33:21.797110 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.797045 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:21.799263 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.799229 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lsdr8\"" Jul 22 04:33:21.799376 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.799356 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 04:33:21.799560 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.799544 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 04:33:21.938220 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.938188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b2d604a-78f3-4926-bbf7-2d605241b70d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tl5d2\" (UID: \"6b2d604a-78f3-4926-bbf7-2d605241b70d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:21.938638 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.938233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbqs8\" (UniqueName: \"kubernetes.io/projected/6b2d604a-78f3-4926-bbf7-2d605241b70d-kube-api-access-mbqs8\") pod \"cert-manager-webhook-6486ffd997-tl5d2\" (UID: \"6b2d604a-78f3-4926-bbf7-2d605241b70d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:21.938638 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.938292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:21.938638 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.938328 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvvjr\" (UniqueName: \"kubernetes.io/projected/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-kube-api-access-wvvjr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:21.938638 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:21.938453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.039105 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.038984 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.039105 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.039046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b2d604a-78f3-4926-bbf7-2d605241b70d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tl5d2\" (UID: \"6b2d604a-78f3-4926-bbf7-2d605241b70d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:22.039406 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.039149 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbqs8\" (UniqueName: \"kubernetes.io/projected/6b2d604a-78f3-4926-bbf7-2d605241b70d-kube-api-access-mbqs8\") pod \"cert-manager-webhook-6486ffd997-tl5d2\" (UID: \"6b2d604a-78f3-4926-bbf7-2d605241b70d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:22.039406 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.039211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.039516 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.039426 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.039571 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.039527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvvjr\" (UniqueName: \"kubernetes.io/projected/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-kube-api-access-wvvjr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.039627 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.039602 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.047163 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.047139 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6b2d604a-78f3-4926-bbf7-2d605241b70d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tl5d2\" (UID: \"6b2d604a-78f3-4926-bbf7-2d605241b70d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:22.047806 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.047778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvvjr\" (UniqueName: \"kubernetes.io/projected/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-kube-api-access-wvvjr\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.047910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.047895 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbqs8\" (UniqueName: \"kubernetes.io/projected/6b2d604a-78f3-4926-bbf7-2d605241b70d-kube-api-access-mbqs8\") pod \"cert-manager-webhook-6486ffd997-tl5d2\" (UID: \"6b2d604a-78f3-4926-bbf7-2d605241b70d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:22.101936 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.101896 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:22.110996 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.110958 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:22.271571 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.271544 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tl5d2"] Jul 22 04:33:22.272945 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:33:22.272913 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b2d604a_78f3_4926_bbf7_2d605241b70d.slice/crio-bd370f8108095bb95052d9bb7a5f2bcfd81c3aee2da20c361d66354031a67e07 WatchSource:0}: Error finding container bd370f8108095bb95052d9bb7a5f2bcfd81c3aee2da20c361d66354031a67e07: Status 404 returned error can't find the container with id bd370f8108095bb95052d9bb7a5f2bcfd81c3aee2da20c361d66354031a67e07 Jul 22 04:33:22.297264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.294924 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g"] Jul 22 04:33:22.300824 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:33:22.300795 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a9afe4_0a3b_46ed_a4ae_a1df41fe731a.slice/crio-67dab916eaca358a5f45bb6521d9b284e68bbf40975d5bc8a77a8fd1466d5b77 WatchSource:0}: Error finding container 67dab916eaca358a5f45bb6521d9b284e68bbf40975d5bc8a77a8fd1466d5b77: Status 404 returned error can't find the container with id 67dab916eaca358a5f45bb6521d9b284e68bbf40975d5bc8a77a8fd1466d5b77 Jul 22 04:33:22.838623 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.838589 2566 generic.go:356] "Generic (PLEG): container finished" podID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerID="9fe15578085212edfee876604f6374d7970bff4e37093e3a9e5f562a8db2debf" exitCode=0 Jul 22 04:33:22.838825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.838679 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" event={"ID":"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a","Type":"ContainerDied","Data":"9fe15578085212edfee876604f6374d7970bff4e37093e3a9e5f562a8db2debf"} Jul 22 04:33:22.838825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.838739 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" event={"ID":"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a","Type":"ContainerStarted","Data":"67dab916eaca358a5f45bb6521d9b284e68bbf40975d5bc8a77a8fd1466d5b77"} Jul 22 04:33:22.839989 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:22.839960 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" event={"ID":"6b2d604a-78f3-4926-bbf7-2d605241b70d","Type":"ContainerStarted","Data":"bd370f8108095bb95052d9bb7a5f2bcfd81c3aee2da20c361d66354031a67e07"} Jul 22 04:33:25.856083 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:25.856026 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" event={"ID":"6b2d604a-78f3-4926-bbf7-2d605241b70d","Type":"ContainerStarted","Data":"e4dc46178cbb8d9157bf432f8b584f9791fcacf681380ddf4ba510807b240f84"} Jul 22 04:33:25.856580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:25.856204 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:25.872751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:25.872706 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" podStartSLOduration=2.177469729 podStartE2EDuration="4.872693232s" totalImagesPullingTime="2.695223503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:25.871252221 +0000 UTC m=+529.960531394" watchObservedRunningTime="2026-07-22 04:33:25.872693232 +0000 UTC m=+529.961972404" Jul 22 04:33:31.863463 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:31.863392 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tl5d2" Jul 22 04:33:35.907257 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:35.907221 2566 generic.go:356] "Generic (PLEG): container finished" podID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerID="4ca856bb56f5660a52999b30470e56f186fd13784f3fa0305ddc3fd60716b39f" exitCode=0 Jul 22 04:33:35.907779 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:35.907304 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" event={"ID":"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a","Type":"ContainerDied","Data":"4ca856bb56f5660a52999b30470e56f186fd13784f3fa0305ddc3fd60716b39f"} Jul 22 04:33:36.913085 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:36.913038 2566 generic.go:356] "Generic (PLEG): container finished" podID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerID="7fbdafb94133e1a0d8ff72bcfd33ec750362095f4f813b25a92fdd2f23c5d8c1" exitCode=0 Jul 22 04:33:36.913398 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:36.913095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" event={"ID":"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a","Type":"ContainerDied","Data":"7fbdafb94133e1a0d8ff72bcfd33ec750362095f4f813b25a92fdd2f23c5d8c1"} Jul 22 04:33:38.042852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.042828 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:38.175185 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.175112 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-util\") pod \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " Jul 22 04:33:38.175185 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.175171 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-bundle\") pod \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " Jul 22 04:33:38.175365 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.175206 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvvjr\" (UniqueName: \"kubernetes.io/projected/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-kube-api-access-wvvjr\") pod \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\" (UID: \"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a\") " Jul 22 04:33:38.175553 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.175525 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-bundle" (OuterVolumeSpecName: "bundle") pod "61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" (UID: "61a9afe4-0a3b-46ed-a4ae-a1df41fe731a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:33:38.177277 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.177255 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-kube-api-access-wvvjr" (OuterVolumeSpecName: "kube-api-access-wvvjr") pod "61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" (UID: "61a9afe4-0a3b-46ed-a4ae-a1df41fe731a"). InnerVolumeSpecName "kube-api-access-wvvjr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:33:38.179490 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.179463 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-util" (OuterVolumeSpecName: "util") pod "61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" (UID: "61a9afe4-0a3b-46ed-a4ae-a1df41fe731a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:33:38.276718 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.276678 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:38.276718 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.276705 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:38.276718 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.276715 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvvjr\" (UniqueName: \"kubernetes.io/projected/61a9afe4-0a3b-46ed-a4ae-a1df41fe731a-kube-api-access-wvvjr\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:38.923361 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.923331 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" Jul 22 04:33:38.923361 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.923336 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fxbq5g" event={"ID":"61a9afe4-0a3b-46ed-a4ae-a1df41fe731a","Type":"ContainerDied","Data":"67dab916eaca358a5f45bb6521d9b284e68bbf40975d5bc8a77a8fd1466d5b77"} Jul 22 04:33:38.923577 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:38.923419 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67dab916eaca358a5f45bb6521d9b284e68bbf40975d5bc8a77a8fd1466d5b77" Jul 22 04:33:45.898745 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.898708 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n"] Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899039 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="pull" Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899050 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="pull" Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899061 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="util" Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899082 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="util" Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899097 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="extract" Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899106 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="extract" Jul 22 04:33:45.899181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.899171 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="61a9afe4-0a3b-46ed-a4ae-a1df41fe731a" containerName="extract" Jul 22 04:33:45.901165 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.901144 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:45.903353 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.903332 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 04:33:45.903452 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.903414 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 04:33:45.904223 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.904203 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-9hmmc\"" Jul 22 04:33:45.912613 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.912592 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n"] Jul 22 04:33:45.936427 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.936399 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6f83e1f-b407-4b86-8d8d-e1a3e93afedc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wbk7n\" (UID: \"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:45.936536 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:45.936428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tbp8\" (UniqueName: \"kubernetes.io/projected/e6f83e1f-b407-4b86-8d8d-e1a3e93afedc-kube-api-access-4tbp8\") pod \"openshift-lws-operator-fd8ccff4c-wbk7n\" (UID: \"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:46.037855 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.037820 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6f83e1f-b407-4b86-8d8d-e1a3e93afedc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wbk7n\" (UID: \"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:46.037855 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.037856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tbp8\" (UniqueName: \"kubernetes.io/projected/e6f83e1f-b407-4b86-8d8d-e1a3e93afedc-kube-api-access-4tbp8\") pod \"openshift-lws-operator-fd8ccff4c-wbk7n\" (UID: \"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:46.038216 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.038198 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e6f83e1f-b407-4b86-8d8d-e1a3e93afedc-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wbk7n\" (UID: \"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:46.045329 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.045306 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tbp8\" (UniqueName: \"kubernetes.io/projected/e6f83e1f-b407-4b86-8d8d-e1a3e93afedc-kube-api-access-4tbp8\") pod \"openshift-lws-operator-fd8ccff4c-wbk7n\" (UID: \"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:46.209835 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.209729 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" Jul 22 04:33:46.336194 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.336006 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n"] Jul 22 04:33:46.339219 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:33:46.339188 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6f83e1f_b407_4b86_8d8d_e1a3e93afedc.slice/crio-8fd38ce5416b6a0176e23cdb43c5a483cb06d9c6d40c2fd7847bb1ff96b553e3 WatchSource:0}: Error finding container 8fd38ce5416b6a0176e23cdb43c5a483cb06d9c6d40c2fd7847bb1ff96b553e3: Status 404 returned error can't find the container with id 8fd38ce5416b6a0176e23cdb43c5a483cb06d9c6d40c2fd7847bb1ff96b553e3 Jul 22 04:33:46.959409 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:46.959376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" event={"ID":"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc","Type":"ContainerStarted","Data":"8fd38ce5416b6a0176e23cdb43c5a483cb06d9c6d40c2fd7847bb1ff96b553e3"} Jul 22 04:33:48.969466 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:48.969433 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" event={"ID":"e6f83e1f-b407-4b86-8d8d-e1a3e93afedc","Type":"ContainerStarted","Data":"8a4d24fd839cbb48753bcab655bbfb333c3fefee306a4c3ac9bf1c75509c9f33"} Jul 22 04:33:48.984648 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:48.984600 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wbk7n" podStartSLOduration=2.329112208 podStartE2EDuration="3.984586962s" totalImagesPullingTime="1.655474754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:33:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:33:48.983293998 +0000 UTC m=+553.072573169" watchObservedRunningTime="2026-07-22 04:33:48.984586962 +0000 UTC m=+553.073866134" Jul 22 04:33:52.596533 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.596497 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt"] Jul 22 04:33:52.599037 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.599021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.601397 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.601373 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 04:33:52.601523 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.601466 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 04:33:52.601523 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.601474 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lsdr8\"" Jul 22 04:33:52.608585 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.608566 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt"] Jul 22 04:33:52.690258 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.690227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nnqt\" (UniqueName: \"kubernetes.io/projected/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-kube-api-access-4nnqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.690408 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.690292 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.690408 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.690365 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.791417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.791376 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nnqt\" (UniqueName: \"kubernetes.io/projected/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-kube-api-access-4nnqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.791583 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.791437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.791583 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.791469 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.791825 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.791807 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.791919 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.791832 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.799339 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.799315 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nnqt\" (UniqueName: \"kubernetes.io/projected/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-kube-api-access-4nnqt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:52.908572 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:52.908544 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:53.049476 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:53.049451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt"] Jul 22 04:33:53.051568 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:33:53.051540 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfb20e7_b56e_41db_91bb_46ad4a3a174b.slice/crio-31b518e30434094cb963a98bedb35ff69745bdbc9795d2c5019502fe409297cd WatchSource:0}: Error finding container 31b518e30434094cb963a98bedb35ff69745bdbc9795d2c5019502fe409297cd: Status 404 returned error can't find the container with id 31b518e30434094cb963a98bedb35ff69745bdbc9795d2c5019502fe409297cd Jul 22 04:33:53.994372 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:53.994336 2566 generic.go:356] "Generic (PLEG): container finished" podID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerID="3dc810242fd3b45d7a26d2f8f07d474e091ec5854ce41c19b5a19fce3203ec02" exitCode=0 Jul 22 04:33:53.994741 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:53.994421 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" event={"ID":"ebfb20e7-b56e-41db-91bb-46ad4a3a174b","Type":"ContainerDied","Data":"3dc810242fd3b45d7a26d2f8f07d474e091ec5854ce41c19b5a19fce3203ec02"} Jul 22 04:33:53.994741 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:53.994472 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" event={"ID":"ebfb20e7-b56e-41db-91bb-46ad4a3a174b","Type":"ContainerStarted","Data":"31b518e30434094cb963a98bedb35ff69745bdbc9795d2c5019502fe409297cd"} Jul 22 04:33:55.000907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:55.000873 2566 generic.go:356] "Generic (PLEG): container finished" podID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerID="9e8ce6dc08ee252bc169b0591cb0e7249d7aff6cc12486930dc60d66193321b9" exitCode=0 Jul 22 04:33:55.001312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:55.000931 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" event={"ID":"ebfb20e7-b56e-41db-91bb-46ad4a3a174b","Type":"ContainerDied","Data":"9e8ce6dc08ee252bc169b0591cb0e7249d7aff6cc12486930dc60d66193321b9"} Jul 22 04:33:56.008374 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:56.008339 2566 generic.go:356] "Generic (PLEG): container finished" podID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerID="56924d25ee8eeb12f97dfa0b838b47112e925d902be3867fa2b3b0917d499dd8" exitCode=0 Jul 22 04:33:56.008788 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:56.008418 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" event={"ID":"ebfb20e7-b56e-41db-91bb-46ad4a3a174b","Type":"ContainerDied","Data":"56924d25ee8eeb12f97dfa0b838b47112e925d902be3867fa2b3b0917d499dd8"} Jul 22 04:33:57.145287 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.145261 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:57.233368 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.233330 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-bundle\") pod \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " Jul 22 04:33:57.233547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.233393 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4nnqt\" (UniqueName: \"kubernetes.io/projected/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-kube-api-access-4nnqt\") pod \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " Jul 22 04:33:57.233547 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.233420 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-util\") pod \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\" (UID: \"ebfb20e7-b56e-41db-91bb-46ad4a3a174b\") " Jul 22 04:33:57.234139 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.234102 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-bundle" (OuterVolumeSpecName: "bundle") pod "ebfb20e7-b56e-41db-91bb-46ad4a3a174b" (UID: "ebfb20e7-b56e-41db-91bb-46ad4a3a174b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:33:57.235477 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.235449 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-kube-api-access-4nnqt" (OuterVolumeSpecName: "kube-api-access-4nnqt") pod "ebfb20e7-b56e-41db-91bb-46ad4a3a174b" (UID: "ebfb20e7-b56e-41db-91bb-46ad4a3a174b"). InnerVolumeSpecName "kube-api-access-4nnqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:33:57.238589 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.238557 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-util" (OuterVolumeSpecName: "util") pod "ebfb20e7-b56e-41db-91bb-46ad4a3a174b" (UID: "ebfb20e7-b56e-41db-91bb-46ad4a3a174b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:33:57.335137 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.335021 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:57.335137 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.335055 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4nnqt\" (UniqueName: \"kubernetes.io/projected/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-kube-api-access-4nnqt\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:57.335137 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:57.335064 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ebfb20e7-b56e-41db-91bb-46ad4a3a174b-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:33:58.018768 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:58.018737 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" Jul 22 04:33:58.018937 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:58.018732 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd7xkt" event={"ID":"ebfb20e7-b56e-41db-91bb-46ad4a3a174b","Type":"ContainerDied","Data":"31b518e30434094cb963a98bedb35ff69745bdbc9795d2c5019502fe409297cd"} Jul 22 04:33:58.018937 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:33:58.018860 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="31b518e30434094cb963a98bedb35ff69745bdbc9795d2c5019502fe409297cd" Jul 22 04:34:03.903128 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903087 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-596745447d-sljz5"] Jul 22 04:34:03.903537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903440 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="util" Jul 22 04:34:03.903537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903452 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="util" Jul 22 04:34:03.903537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903467 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="pull" Jul 22 04:34:03.903537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903474 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="pull" Jul 22 04:34:03.903537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903492 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="extract" Jul 22 04:34:03.903537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903501 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="extract" Jul 22 04:34:03.903747 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.903554 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfb20e7-b56e-41db-91bb-46ad4a3a174b" containerName="extract" Jul 22 04:34:03.906485 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.906461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:03.909724 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.909698 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 04:34:03.909843 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.909763 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wdtv9\"" Jul 22 04:34:03.909843 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.909786 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 04:34:03.909940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.909849 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 04:34:03.923470 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.923446 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-596745447d-sljz5"] Jul 22 04:34:03.989795 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.989760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-cert\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:03.989979 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.989802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-metrics-cert\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:03.989979 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.989855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-manager-config\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:03.990094 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:03.989979 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g26lh\" (UniqueName: \"kubernetes.io/projected/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-kube-api-access-g26lh\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.090692 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.090661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g26lh\" (UniqueName: \"kubernetes.io/projected/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-kube-api-access-g26lh\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.090870 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.090713 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-cert\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.090870 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.090736 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-metrics-cert\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.090870 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.090763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-manager-config\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.091617 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.091583 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-manager-config\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.093204 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.093183 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-cert\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.093304 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.093252 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-metrics-cert\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.098684 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.098664 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g26lh\" (UniqueName: \"kubernetes.io/projected/9714745a-4e1e-4bb3-afdf-8d9d84d09e0b-kube-api-access-g26lh\") pod \"lws-controller-manager-596745447d-sljz5\" (UID: \"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b\") " pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.216642 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.216559 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:04.360772 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.360749 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-596745447d-sljz5"] Jul 22 04:34:04.362529 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:34:04.362502 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9714745a_4e1e_4bb3_afdf_8d9d84d09e0b.slice/crio-43486da7145b9daa49223eeaadc646f940ff32b3a8b2196febb96a0e9adbe1d0 WatchSource:0}: Error finding container 43486da7145b9daa49223eeaadc646f940ff32b3a8b2196febb96a0e9adbe1d0: Status 404 returned error can't find the container with id 43486da7145b9daa49223eeaadc646f940ff32b3a8b2196febb96a0e9adbe1d0 Jul 22 04:34:04.882977 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.882937 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s"] Jul 22 04:34:04.887593 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.887564 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:04.889835 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.889813 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 04:34:04.890084 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.890044 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 04:34:04.890191 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.890094 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vs5b6\"" Jul 22 04:34:04.890344 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.890325 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 04:34:04.890398 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.890348 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 04:34:04.903441 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.903412 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s"] Jul 22 04:34:04.998498 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.998460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rjs4\" (UniqueName: \"kubernetes.io/projected/86342374-e0bc-46ce-8b6c-6563caab986c-kube-api-access-5rjs4\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:04.998664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.998503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86342374-e0bc-46ce-8b6c-6563caab986c-manifests\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:04.998664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.998586 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86342374-e0bc-46ce-8b6c-6563caab986c-tmp\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:04.998664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.998630 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86342374-e0bc-46ce-8b6c-6563caab986c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:04.998664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:04.998652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86342374-e0bc-46ce-8b6c-6563caab986c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.052827 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.052793 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" event={"ID":"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b","Type":"ContainerStarted","Data":"43486da7145b9daa49223eeaadc646f940ff32b3a8b2196febb96a0e9adbe1d0"} Jul 22 04:34:05.099766 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.099710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86342374-e0bc-46ce-8b6c-6563caab986c-tmp\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.099945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.099774 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86342374-e0bc-46ce-8b6c-6563caab986c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.099945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.099805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86342374-e0bc-46ce-8b6c-6563caab986c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.099945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.099881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rjs4\" (UniqueName: \"kubernetes.io/projected/86342374-e0bc-46ce-8b6c-6563caab986c-kube-api-access-5rjs4\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.099945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.099915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86342374-e0bc-46ce-8b6c-6563caab986c-manifests\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.100408 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.100380 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/86342374-e0bc-46ce-8b6c-6563caab986c-manifests\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.102651 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.102620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86342374-e0bc-46ce-8b6c-6563caab986c-tmp\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.103043 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.102873 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/86342374-e0bc-46ce-8b6c-6563caab986c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.103323 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.103300 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/86342374-e0bc-46ce-8b6c-6563caab986c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.121837 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.121806 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rjs4\" (UniqueName: \"kubernetes.io/projected/86342374-e0bc-46ce-8b6c-6563caab986c-kube-api-access-5rjs4\") pod \"opendatahub-operator-controller-manager-5676cbf8df-t9j2s\" (UID: \"86342374-e0bc-46ce-8b6c-6563caab986c\") " pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.198083 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.198039 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:05.359955 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:05.359589 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s"] Jul 22 04:34:05.363188 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:34:05.363157 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86342374_e0bc_46ce_8b6c_6563caab986c.slice/crio-ffa3cfd5638fb6adf37cfa2da1bb6dc150b204a267ec79833e4bd5705255b4b5 WatchSource:0}: Error finding container ffa3cfd5638fb6adf37cfa2da1bb6dc150b204a267ec79833e4bd5705255b4b5: Status 404 returned error can't find the container with id ffa3cfd5638fb6adf37cfa2da1bb6dc150b204a267ec79833e4bd5705255b4b5 Jul 22 04:34:06.060509 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:06.060459 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" event={"ID":"9714745a-4e1e-4bb3-afdf-8d9d84d09e0b","Type":"ContainerStarted","Data":"4d7e3d6aea3cfd1c8522d3ea865c5fbc92801d59edba8930f37cc3ba28b95a11"} Jul 22 04:34:06.060934 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:06.060542 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:06.062023 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:06.062000 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" event={"ID":"86342374-e0bc-46ce-8b6c-6563caab986c","Type":"ContainerStarted","Data":"ffa3cfd5638fb6adf37cfa2da1bb6dc150b204a267ec79833e4bd5705255b4b5"} Jul 22 04:34:06.078555 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:06.078510 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" podStartSLOduration=1.5317490390000001 podStartE2EDuration="3.078495572s" totalImagesPullingTime="1.546746533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:06.076717002 +0000 UTC m=+570.165996225" watchObservedRunningTime="2026-07-22 04:34:06.078495572 +0000 UTC m=+570.167774743" Jul 22 04:34:09.080987 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:09.080947 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" event={"ID":"86342374-e0bc-46ce-8b6c-6563caab986c","Type":"ContainerStarted","Data":"8432b7a8e99d6ae148785526c12d7974c3b717bf4c0db29d9ef0cb7dc525e040"} Jul 22 04:34:10.088186 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:10.088157 2566 generic.go:356] "Generic (PLEG): container finished" podID="86342374-e0bc-46ce-8b6c-6563caab986c" containerID="8432b7a8e99d6ae148785526c12d7974c3b717bf4c0db29d9ef0cb7dc525e040" exitCode=0 Jul 22 04:34:10.088580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:10.088197 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" event={"ID":"86342374-e0bc-46ce-8b6c-6563caab986c","Type":"ContainerDied","Data":"8432b7a8e99d6ae148785526c12d7974c3b717bf4c0db29d9ef0cb7dc525e040"} Jul 22 04:34:11.094786 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:11.094676 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" event={"ID":"86342374-e0bc-46ce-8b6c-6563caab986c","Type":"ContainerStarted","Data":"9370fef7f20e160da880430f37077f909afa41c04e7e3f9dde4f2beb5e5ff3a3"} Jul 22 04:34:11.094786 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:11.094778 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:11.113302 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:11.113254 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" podStartSLOduration=2.302961457 podStartE2EDuration="7.113237603s" totalImagesPullingTime="3.793995659s" totalInitContainerRuntime="1.016280487s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:04 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:11.11201705 +0000 UTC m=+575.201296224" watchObservedRunningTime="2026-07-22 04:34:11.113237603 +0000 UTC m=+575.202516776" Jul 22 04:34:17.070874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:17.070846 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-596745447d-sljz5" Jul 22 04:34:22.101222 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:22.101194 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5676cbf8df-t9j2s" Jul 22 04:34:25.182725 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.182689 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86559b66b8-w77r2"] Jul 22 04:34:25.193173 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.193149 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.196549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.196519 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 22 04:34:25.196682 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.196570 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 22 04:34:25.198003 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.197962 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-cp2gc\"" Jul 22 04:34:25.198739 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.198646 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86559b66b8-w77r2"] Jul 22 04:34:25.203318 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.203290 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 04:34:25.208984 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.208964 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 04:34:25.278077 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.278047 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f12fd811-9472-430a-bd16-02d18b44aae5-tmp\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.278208 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.278155 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12fd811-9472-430a-bd16-02d18b44aae5-tls-certs\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.278208 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.278175 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26rk\" (UniqueName: \"kubernetes.io/projected/f12fd811-9472-430a-bd16-02d18b44aae5-kube-api-access-m26rk\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.379249 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.379216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12fd811-9472-430a-bd16-02d18b44aae5-tls-certs\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.379408 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.379256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26rk\" (UniqueName: \"kubernetes.io/projected/f12fd811-9472-430a-bd16-02d18b44aae5-kube-api-access-m26rk\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.379408 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.379294 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f12fd811-9472-430a-bd16-02d18b44aae5-tmp\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.379408 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:34:25.379369 2566 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 22 04:34:25.379538 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:34:25.379453 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f12fd811-9472-430a-bd16-02d18b44aae5-tls-certs podName:f12fd811-9472-430a-bd16-02d18b44aae5 nodeName:}" failed. No retries permitted until 2026-07-22 04:34:25.879431823 +0000 UTC m=+589.968710987 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f12fd811-9472-430a-bd16-02d18b44aae5-tls-certs") pod "kube-auth-proxy-86559b66b8-w77r2" (UID: "f12fd811-9472-430a-bd16-02d18b44aae5") : secret "kube-auth-proxy-tls" not found Jul 22 04:34:25.381528 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.381507 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f12fd811-9472-430a-bd16-02d18b44aae5-tmp\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.388300 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.388281 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26rk\" (UniqueName: \"kubernetes.io/projected/f12fd811-9472-430a-bd16-02d18b44aae5-kube-api-access-m26rk\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.882783 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.882739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12fd811-9472-430a-bd16-02d18b44aae5-tls-certs\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:25.885267 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:25.885241 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f12fd811-9472-430a-bd16-02d18b44aae5-tls-certs\") pod \"kube-auth-proxy-86559b66b8-w77r2\" (UID: \"f12fd811-9472-430a-bd16-02d18b44aae5\") " pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:26.105436 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:26.105394 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" Jul 22 04:34:26.242311 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:26.242281 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86559b66b8-w77r2"] Jul 22 04:34:26.245329 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:34:26.245305 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12fd811_9472_430a_bd16_02d18b44aae5.slice/crio-dda2f4b67ad4593cbff6016d4dc8006ae2c915e0ade291eaeb1928c9a474361f WatchSource:0}: Error finding container dda2f4b67ad4593cbff6016d4dc8006ae2c915e0ade291eaeb1928c9a474361f: Status 404 returned error can't find the container with id dda2f4b67ad4593cbff6016d4dc8006ae2c915e0ade291eaeb1928c9a474361f Jul 22 04:34:27.176209 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:27.176174 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" event={"ID":"f12fd811-9472-430a-bd16-02d18b44aae5","Type":"ContainerStarted","Data":"dda2f4b67ad4593cbff6016d4dc8006ae2c915e0ade291eaeb1928c9a474361f"} Jul 22 04:34:29.189556 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:29.189518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" event={"ID":"f12fd811-9472-430a-bd16-02d18b44aae5","Type":"ContainerStarted","Data":"2a08c948b42f3773959bcfbcdd5b45dd6052850e7b8e880e0268fc45e0ec8bce"} Jul 22 04:34:29.206826 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:34:29.206777 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86559b66b8-w77r2" podStartSLOduration=1.768131333 podStartE2EDuration="4.20675956s" totalImagesPullingTime="2.438628227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:34:29.203726073 +0000 UTC m=+593.293005254" watchObservedRunningTime="2026-07-22 04:34:29.20675956 +0000 UTC m=+593.296038731" Jul 22 04:35:41.648997 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.648964 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8"] Jul 22 04:35:41.653082 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.653046 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.655295 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.655273 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vrt4d\"" Jul 22 04:35:41.655295 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.655286 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 04:35:41.655923 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.655907 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 04:35:41.660772 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.660750 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8"] Jul 22 04:35:41.759048 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.759002 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.759238 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.759055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd8xl\" (UniqueName: \"kubernetes.io/projected/484cc7ac-b16a-4f06-b551-bbb7039f28e0-kube-api-access-qd8xl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.759238 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.759150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.860214 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.860175 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.860393 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.860219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd8xl\" (UniqueName: \"kubernetes.io/projected/484cc7ac-b16a-4f06-b551-bbb7039f28e0-kube-api-access-qd8xl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.860393 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.860290 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.860653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.860633 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.860653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.860645 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.868556 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.868529 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd8xl\" (UniqueName: \"kubernetes.io/projected/484cc7ac-b16a-4f06-b551-bbb7039f28e0-kube-api-access-qd8xl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:41.963484 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:41.963397 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:42.100282 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.100250 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8"] Jul 22 04:35:42.102594 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:35:42.102562 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod484cc7ac_b16a_4f06_b551_bbb7039f28e0.slice/crio-f6facff27ce6f1138bf6f18d7e4ea9ea70718fe2a717ba94aad29609c75e3660 WatchSource:0}: Error finding container f6facff27ce6f1138bf6f18d7e4ea9ea70718fe2a717ba94aad29609c75e3660: Status 404 returned error can't find the container with id f6facff27ce6f1138bf6f18d7e4ea9ea70718fe2a717ba94aad29609c75e3660 Jul 22 04:35:42.254290 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.254212 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx"] Jul 22 04:35:42.257895 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.257872 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.265764 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.265741 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx"] Jul 22 04:35:42.365567 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.365535 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.365740 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.365578 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lfmg\" (UniqueName: \"kubernetes.io/projected/960943a7-a739-4df1-a4ed-f92a13c4e1d3-kube-api-access-7lfmg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.365740 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.365697 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.466872 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.466834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.467045 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.466885 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.467045 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.466921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lfmg\" (UniqueName: \"kubernetes.io/projected/960943a7-a739-4df1-a4ed-f92a13c4e1d3-kube-api-access-7lfmg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.467347 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.467322 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.467347 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.467338 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.474864 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.474843 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lfmg\" (UniqueName: \"kubernetes.io/projected/960943a7-a739-4df1-a4ed-f92a13c4e1d3-kube-api-access-7lfmg\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.542746 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.542665 2566 generic.go:356] "Generic (PLEG): container finished" podID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerID="fb3729bd5e2ec01a2f777ddb4986e8ab1e11ddd87872547cda27f86c236826b4" exitCode=0 Jul 22 04:35:42.542882 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.542759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" event={"ID":"484cc7ac-b16a-4f06-b551-bbb7039f28e0","Type":"ContainerDied","Data":"fb3729bd5e2ec01a2f777ddb4986e8ab1e11ddd87872547cda27f86c236826b4"} Jul 22 04:35:42.542882 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.542810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" event={"ID":"484cc7ac-b16a-4f06-b551-bbb7039f28e0","Type":"ContainerStarted","Data":"f6facff27ce6f1138bf6f18d7e4ea9ea70718fe2a717ba94aad29609c75e3660"} Jul 22 04:35:42.543198 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.543183 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:35:42.570612 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.570587 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:42.701364 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.701332 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx"] Jul 22 04:35:42.704610 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:35:42.704577 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960943a7_a739_4df1_a4ed_f92a13c4e1d3.slice/crio-6e4317d4824d1d583cee95c865cb02fb2b2ba4a20d866155110b6fe0b4d6074b WatchSource:0}: Error finding container 6e4317d4824d1d583cee95c865cb02fb2b2ba4a20d866155110b6fe0b4d6074b: Status 404 returned error can't find the container with id 6e4317d4824d1d583cee95c865cb02fb2b2ba4a20d866155110b6fe0b4d6074b Jul 22 04:35:42.852668 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.852635 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q"] Jul 22 04:35:42.856280 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.856264 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:42.863713 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.863685 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q"] Jul 22 04:35:42.971604 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.971569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:42.971784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.971639 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnmlj\" (UniqueName: \"kubernetes.io/projected/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-kube-api-access-hnmlj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:42.971784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:42.971669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.072509 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.072418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.072509 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.072493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnmlj\" (UniqueName: \"kubernetes.io/projected/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-kube-api-access-hnmlj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.072711 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.072539 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.072889 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.072870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.072943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.072926 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.081512 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.081488 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnmlj\" (UniqueName: \"kubernetes.io/projected/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-kube-api-access-hnmlj\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.191332 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.191294 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:43.252166 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.252129 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj"] Jul 22 04:35:43.256144 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.256022 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.267507 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.266873 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj"] Jul 22 04:35:43.347544 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.347516 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q"] Jul 22 04:35:43.349022 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:35:43.349000 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d3efef_0a3e_43d9_958a_7b0545e4c5a9.slice/crio-0f8357fd083458c3c313f4d14f8fbf7bed391abaa3f228914f92d4467a7f6e16 WatchSource:0}: Error finding container 0f8357fd083458c3c313f4d14f8fbf7bed391abaa3f228914f92d4467a7f6e16: Status 404 returned error can't find the container with id 0f8357fd083458c3c313f4d14f8fbf7bed391abaa3f228914f92d4467a7f6e16 Jul 22 04:35:43.375401 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.375368 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.375401 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.375403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ns8k\" (UniqueName: \"kubernetes.io/projected/a23533ca-695d-4ae4-a997-283dae661255-kube-api-access-5ns8k\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.375624 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.375481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.475951 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.475914 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.476157 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.476004 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.476157 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.476027 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ns8k\" (UniqueName: \"kubernetes.io/projected/a23533ca-695d-4ae4-a997-283dae661255-kube-api-access-5ns8k\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.476312 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.476289 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.476387 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.476374 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.489241 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.489203 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ns8k\" (UniqueName: \"kubernetes.io/projected/a23533ca-695d-4ae4-a997-283dae661255-kube-api-access-5ns8k\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.549658 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.549625 2566 generic.go:356] "Generic (PLEG): container finished" podID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerID="9b0eba25d861680cbb777c169933123a9bd75d24360921866157330987ada8db" exitCode=0 Jul 22 04:35:43.549841 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.549706 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" event={"ID":"484cc7ac-b16a-4f06-b551-bbb7039f28e0","Type":"ContainerDied","Data":"9b0eba25d861680cbb777c169933123a9bd75d24360921866157330987ada8db"} Jul 22 04:35:43.551335 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.551225 2566 generic.go:356] "Generic (PLEG): container finished" podID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerID="1133f26e240cfde93c33b0e0a58dffdd5ff626fba40049ab5f662db660b38c5b" exitCode=0 Jul 22 04:35:43.551335 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.551298 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" event={"ID":"960943a7-a739-4df1-a4ed-f92a13c4e1d3","Type":"ContainerDied","Data":"1133f26e240cfde93c33b0e0a58dffdd5ff626fba40049ab5f662db660b38c5b"} Jul 22 04:35:43.551443 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.551342 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" event={"ID":"960943a7-a739-4df1-a4ed-f92a13c4e1d3","Type":"ContainerStarted","Data":"6e4317d4824d1d583cee95c865cb02fb2b2ba4a20d866155110b6fe0b4d6074b"} Jul 22 04:35:43.552855 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.552832 2566 generic.go:356] "Generic (PLEG): container finished" podID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerID="eeae3a69e6a763eecfe92d4920164e2a0fc3b2ecd0ea2d6bdd982f6380703ca3" exitCode=0 Jul 22 04:35:43.552974 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.552910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" event={"ID":"21d3efef-0a3e-43d9-958a-7b0545e4c5a9","Type":"ContainerDied","Data":"eeae3a69e6a763eecfe92d4920164e2a0fc3b2ecd0ea2d6bdd982f6380703ca3"} Jul 22 04:35:43.552974 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.552947 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" event={"ID":"21d3efef-0a3e-43d9-958a-7b0545e4c5a9","Type":"ContainerStarted","Data":"0f8357fd083458c3c313f4d14f8fbf7bed391abaa3f228914f92d4467a7f6e16"} Jul 22 04:35:43.573868 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.573842 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:43.716092 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:43.716051 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj"] Jul 22 04:35:43.752994 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:35:43.752956 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23533ca_695d_4ae4_a997_283dae661255.slice/crio-befc66accc635c8edb2db0d4e97982908c6e11f2a636982c677a437cbce28709 WatchSource:0}: Error finding container befc66accc635c8edb2db0d4e97982908c6e11f2a636982c677a437cbce28709: Status 404 returned error can't find the container with id befc66accc635c8edb2db0d4e97982908c6e11f2a636982c677a437cbce28709 Jul 22 04:35:44.560328 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.560248 2566 generic.go:356] "Generic (PLEG): container finished" podID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerID="af6a68d45c56f8ccda548f04f2aebdf09feeb1e0ee23983448d8c7552ff4379a" exitCode=0 Jul 22 04:35:44.560472 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.560326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" event={"ID":"21d3efef-0a3e-43d9-958a-7b0545e4c5a9","Type":"ContainerDied","Data":"af6a68d45c56f8ccda548f04f2aebdf09feeb1e0ee23983448d8c7552ff4379a"} Jul 22 04:35:44.561846 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.561825 2566 generic.go:356] "Generic (PLEG): container finished" podID="a23533ca-695d-4ae4-a997-283dae661255" containerID="4ee106799097e6e404cff5abb6c52a47ecfc98a17777e6d0829f49050022ac66" exitCode=0 Jul 22 04:35:44.561922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.561903 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" event={"ID":"a23533ca-695d-4ae4-a997-283dae661255","Type":"ContainerDied","Data":"4ee106799097e6e404cff5abb6c52a47ecfc98a17777e6d0829f49050022ac66"} Jul 22 04:35:44.561969 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.561947 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" event={"ID":"a23533ca-695d-4ae4-a997-283dae661255","Type":"ContainerStarted","Data":"befc66accc635c8edb2db0d4e97982908c6e11f2a636982c677a437cbce28709"} Jul 22 04:35:44.564500 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.564479 2566 generic.go:356] "Generic (PLEG): container finished" podID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerID="055962b25aeb039b579dceb7ff3ea52431ef633bc168b0a40f4fd48f8dc2b6ba" exitCode=0 Jul 22 04:35:44.564585 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.564512 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" event={"ID":"484cc7ac-b16a-4f06-b551-bbb7039f28e0","Type":"ContainerDied","Data":"055962b25aeb039b579dceb7ff3ea52431ef633bc168b0a40f4fd48f8dc2b6ba"} Jul 22 04:35:44.566443 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.566411 2566 generic.go:356] "Generic (PLEG): container finished" podID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerID="9366731e2fc777479c608a5e6c06c022663306aaf0f273801e58936cbfb73925" exitCode=0 Jul 22 04:35:44.566540 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:44.566449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" event={"ID":"960943a7-a739-4df1-a4ed-f92a13c4e1d3","Type":"ContainerDied","Data":"9366731e2fc777479c608a5e6c06c022663306aaf0f273801e58936cbfb73925"} Jul 22 04:35:45.574271 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.574232 2566 generic.go:356] "Generic (PLEG): container finished" podID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerID="44fcdc7ceca4bf0e052bb8dab93856db74f5ff3c3868eb273f9c7253d40b2d3b" exitCode=0 Jul 22 04:35:45.574707 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.574271 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" event={"ID":"960943a7-a739-4df1-a4ed-f92a13c4e1d3","Type":"ContainerDied","Data":"44fcdc7ceca4bf0e052bb8dab93856db74f5ff3c3868eb273f9c7253d40b2d3b"} Jul 22 04:35:45.576276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.576252 2566 generic.go:356] "Generic (PLEG): container finished" podID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerID="af542dbbf2f9516146c0e1c1ccd36510f145a462c33f19eb16074d3b6990f79f" exitCode=0 Jul 22 04:35:45.576380 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.576322 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" event={"ID":"21d3efef-0a3e-43d9-958a-7b0545e4c5a9","Type":"ContainerDied","Data":"af542dbbf2f9516146c0e1c1ccd36510f145a462c33f19eb16074d3b6990f79f"} Jul 22 04:35:45.578038 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.578019 2566 generic.go:356] "Generic (PLEG): container finished" podID="a23533ca-695d-4ae4-a997-283dae661255" containerID="40b6cea0693bccc25aee7b54f35571bae4b7959aafaafa5094e58429cea3c699" exitCode=0 Jul 22 04:35:45.578124 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.578104 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" event={"ID":"a23533ca-695d-4ae4-a997-283dae661255","Type":"ContainerDied","Data":"40b6cea0693bccc25aee7b54f35571bae4b7959aafaafa5094e58429cea3c699"} Jul 22 04:35:45.716445 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.716414 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:45.803643 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.803606 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-util\") pod \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " Jul 22 04:35:45.803826 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.803704 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-bundle\") pod \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " Jul 22 04:35:45.803826 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.803748 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qd8xl\" (UniqueName: \"kubernetes.io/projected/484cc7ac-b16a-4f06-b551-bbb7039f28e0-kube-api-access-qd8xl\") pod \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\" (UID: \"484cc7ac-b16a-4f06-b551-bbb7039f28e0\") " Jul 22 04:35:45.804199 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.804165 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-bundle" (OuterVolumeSpecName: "bundle") pod "484cc7ac-b16a-4f06-b551-bbb7039f28e0" (UID: "484cc7ac-b16a-4f06-b551-bbb7039f28e0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:45.805997 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.805971 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/484cc7ac-b16a-4f06-b551-bbb7039f28e0-kube-api-access-qd8xl" (OuterVolumeSpecName: "kube-api-access-qd8xl") pod "484cc7ac-b16a-4f06-b551-bbb7039f28e0" (UID: "484cc7ac-b16a-4f06-b551-bbb7039f28e0"). InnerVolumeSpecName "kube-api-access-qd8xl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:35:45.808654 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.808633 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-util" (OuterVolumeSpecName: "util") pod "484cc7ac-b16a-4f06-b551-bbb7039f28e0" (UID: "484cc7ac-b16a-4f06-b551-bbb7039f28e0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:45.904546 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.904515 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:45.904546 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.904546 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/484cc7ac-b16a-4f06-b551-bbb7039f28e0-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:45.904716 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:45.904556 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qd8xl\" (UniqueName: \"kubernetes.io/projected/484cc7ac-b16a-4f06-b551-bbb7039f28e0-kube-api-access-qd8xl\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:46.585123 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.585086 2566 generic.go:356] "Generic (PLEG): container finished" podID="a23533ca-695d-4ae4-a997-283dae661255" containerID="daa79846de98b8a39036c905552dd5336f10f7aa043341d3cf9bac4d1d86748c" exitCode=0 Jul 22 04:35:46.585542 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.585169 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" event={"ID":"a23533ca-695d-4ae4-a997-283dae661255","Type":"ContainerDied","Data":"daa79846de98b8a39036c905552dd5336f10f7aa043341d3cf9bac4d1d86748c"} Jul 22 04:35:46.587017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.586992 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" event={"ID":"484cc7ac-b16a-4f06-b551-bbb7039f28e0","Type":"ContainerDied","Data":"f6facff27ce6f1138bf6f18d7e4ea9ea70718fe2a717ba94aad29609c75e3660"} Jul 22 04:35:46.587138 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.587020 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8" Jul 22 04:35:46.587138 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.587031 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6facff27ce6f1138bf6f18d7e4ea9ea70718fe2a717ba94aad29609c75e3660" Jul 22 04:35:46.744863 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.744842 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:46.746795 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.746774 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:46.914160 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914125 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-bundle\") pod \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " Jul 22 04:35:46.914320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914187 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-bundle\") pod \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " Jul 22 04:35:46.914320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914227 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnmlj\" (UniqueName: \"kubernetes.io/projected/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-kube-api-access-hnmlj\") pod \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " Jul 22 04:35:46.914320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914289 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-util\") pod \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " Jul 22 04:35:46.914474 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914341 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-util\") pod \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\" (UID: \"21d3efef-0a3e-43d9-958a-7b0545e4c5a9\") " Jul 22 04:35:46.914474 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914370 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lfmg\" (UniqueName: \"kubernetes.io/projected/960943a7-a739-4df1-a4ed-f92a13c4e1d3-kube-api-access-7lfmg\") pod \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\" (UID: \"960943a7-a739-4df1-a4ed-f92a13c4e1d3\") " Jul 22 04:35:46.914753 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914718 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-bundle" (OuterVolumeSpecName: "bundle") pod "21d3efef-0a3e-43d9-958a-7b0545e4c5a9" (UID: "21d3efef-0a3e-43d9-958a-7b0545e4c5a9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:46.914915 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.914886 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-bundle" (OuterVolumeSpecName: "bundle") pod "960943a7-a739-4df1-a4ed-f92a13c4e1d3" (UID: "960943a7-a739-4df1-a4ed-f92a13c4e1d3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:46.916622 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.916581 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/960943a7-a739-4df1-a4ed-f92a13c4e1d3-kube-api-access-7lfmg" (OuterVolumeSpecName: "kube-api-access-7lfmg") pod "960943a7-a739-4df1-a4ed-f92a13c4e1d3" (UID: "960943a7-a739-4df1-a4ed-f92a13c4e1d3"). InnerVolumeSpecName "kube-api-access-7lfmg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:35:46.916849 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.916826 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-kube-api-access-hnmlj" (OuterVolumeSpecName: "kube-api-access-hnmlj") pod "21d3efef-0a3e-43d9-958a-7b0545e4c5a9" (UID: "21d3efef-0a3e-43d9-958a-7b0545e4c5a9"). InnerVolumeSpecName "kube-api-access-hnmlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:35:46.919751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.919723 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-util" (OuterVolumeSpecName: "util") pod "21d3efef-0a3e-43d9-958a-7b0545e4c5a9" (UID: "21d3efef-0a3e-43d9-958a-7b0545e4c5a9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:46.920919 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:46.920894 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-util" (OuterVolumeSpecName: "util") pod "960943a7-a739-4df1-a4ed-f92a13c4e1d3" (UID: "960943a7-a739-4df1-a4ed-f92a13c4e1d3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:47.014992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.014956 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:47.014992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.014985 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:47.014992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.014994 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnmlj\" (UniqueName: \"kubernetes.io/projected/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-kube-api-access-hnmlj\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:47.014992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.015002 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960943a7-a739-4df1-a4ed-f92a13c4e1d3-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:47.015268 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.015012 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21d3efef-0a3e-43d9-958a-7b0545e4c5a9-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:47.015268 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.015021 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lfmg\" (UniqueName: \"kubernetes.io/projected/960943a7-a739-4df1-a4ed-f92a13c4e1d3-kube-api-access-7lfmg\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:47.594104 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.594056 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" Jul 22 04:35:47.594104 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.594092 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q" event={"ID":"21d3efef-0a3e-43d9-958a-7b0545e4c5a9","Type":"ContainerDied","Data":"0f8357fd083458c3c313f4d14f8fbf7bed391abaa3f228914f92d4467a7f6e16"} Jul 22 04:35:47.594554 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.594139 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f8357fd083458c3c313f4d14f8fbf7bed391abaa3f228914f92d4467a7f6e16" Jul 22 04:35:47.596201 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.596149 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" event={"ID":"960943a7-a739-4df1-a4ed-f92a13c4e1d3","Type":"ContainerDied","Data":"6e4317d4824d1d583cee95c865cb02fb2b2ba4a20d866155110b6fe0b4d6074b"} Jul 22 04:35:47.596309 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.596221 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e4317d4824d1d583cee95c865cb02fb2b2ba4a20d866155110b6fe0b4d6074b" Jul 22 04:35:47.596361 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.596320 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx" Jul 22 04:35:47.739324 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.739300 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:47.921730 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.921682 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ns8k\" (UniqueName: \"kubernetes.io/projected/a23533ca-695d-4ae4-a997-283dae661255-kube-api-access-5ns8k\") pod \"a23533ca-695d-4ae4-a997-283dae661255\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " Jul 22 04:35:47.921924 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.921745 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-util\") pod \"a23533ca-695d-4ae4-a997-283dae661255\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " Jul 22 04:35:47.921924 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.921786 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-bundle\") pod \"a23533ca-695d-4ae4-a997-283dae661255\" (UID: \"a23533ca-695d-4ae4-a997-283dae661255\") " Jul 22 04:35:47.922343 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.922316 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-bundle" (OuterVolumeSpecName: "bundle") pod "a23533ca-695d-4ae4-a997-283dae661255" (UID: "a23533ca-695d-4ae4-a997-283dae661255"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:47.923874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.923843 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a23533ca-695d-4ae4-a997-283dae661255-kube-api-access-5ns8k" (OuterVolumeSpecName: "kube-api-access-5ns8k") pod "a23533ca-695d-4ae4-a997-283dae661255" (UID: "a23533ca-695d-4ae4-a997-283dae661255"). InnerVolumeSpecName "kube-api-access-5ns8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:35:47.926671 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:47.926650 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-util" (OuterVolumeSpecName: "util") pod "a23533ca-695d-4ae4-a997-283dae661255" (UID: "a23533ca-695d-4ae4-a997-283dae661255"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:35:48.023168 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:48.023138 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:48.023168 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:48.023166 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ns8k\" (UniqueName: \"kubernetes.io/projected/a23533ca-695d-4ae4-a997-283dae661255-kube-api-access-5ns8k\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:48.023168 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:48.023176 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a23533ca-695d-4ae4-a997-283dae661255-util\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:35:48.602889 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:48.602859 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" Jul 22 04:35:48.602889 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:48.602871 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj" event={"ID":"a23533ca-695d-4ae4-a997-283dae661255","Type":"ContainerDied","Data":"befc66accc635c8edb2db0d4e97982908c6e11f2a636982c677a437cbce28709"} Jul 22 04:35:48.603390 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:48.602913 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="befc66accc635c8edb2db0d4e97982908c6e11f2a636982c677a437cbce28709" Jul 22 04:35:51.573295 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573255 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58578f969f-ndgrz"] Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573591 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="util" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573603 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="util" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573616 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="pull" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573621 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="pull" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573627 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="extract" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573633 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="extract" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573639 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="util" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573644 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="util" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573650 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="pull" Jul 22 04:35:51.573653 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573655 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="pull" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573667 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="util" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573672 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="util" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573678 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573683 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573690 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="util" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573695 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="util" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573703 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573708 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573715 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573721 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573732 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="pull" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573737 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="pull" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573744 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="pull" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573749 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="pull" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573801 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="484cc7ac-b16a-4f06-b551-bbb7039f28e0" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573809 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d3efef-0a3e-43d9-958a-7b0545e4c5a9" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573815 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="960943a7-a739-4df1-a4ed-f92a13c4e1d3" containerName="extract" Jul 22 04:35:51.573943 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.573824 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a23533ca-695d-4ae4-a997-283dae661255" containerName="extract" Jul 22 04:35:51.577134 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.577115 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.588358 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.588335 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58578f969f-ndgrz"] Jul 22 04:35:51.754417 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754373 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e16db6-39ea-4c19-8125-f69b9a315485-console-oauth-config\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.754600 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-console-config\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.754600 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-service-ca\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.754600 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754493 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ld5v\" (UniqueName: \"kubernetes.io/projected/68e16db6-39ea-4c19-8125-f69b9a315485-kube-api-access-8ld5v\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.754600 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754547 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-oauth-serving-cert\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.754600 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e16db6-39ea-4c19-8125-f69b9a315485-console-serving-cert\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.754806 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.754601 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-trusted-ca-bundle\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.855685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855599 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-console-config\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.855685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855632 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-service-ca\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.855685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ld5v\" (UniqueName: \"kubernetes.io/projected/68e16db6-39ea-4c19-8125-f69b9a315485-kube-api-access-8ld5v\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.855685 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-oauth-serving-cert\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855705 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e16db6-39ea-4c19-8125-f69b9a315485-console-serving-cert\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-trusted-ca-bundle\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856030 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.855783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e16db6-39ea-4c19-8125-f69b9a315485-console-oauth-config\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856464 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.856433 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-service-ca\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856550 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.856522 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-console-config\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856593 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.856572 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-oauth-serving-cert\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.856718 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.856700 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68e16db6-39ea-4c19-8125-f69b9a315485-trusted-ca-bundle\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.858197 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.858176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/68e16db6-39ea-4c19-8125-f69b9a315485-console-oauth-config\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.858283 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.858265 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/68e16db6-39ea-4c19-8125-f69b9a315485-console-serving-cert\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.863412 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.863391 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ld5v\" (UniqueName: \"kubernetes.io/projected/68e16db6-39ea-4c19-8125-f69b9a315485-kube-api-access-8ld5v\") pod \"console-58578f969f-ndgrz\" (UID: \"68e16db6-39ea-4c19-8125-f69b9a315485\") " pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:51.886331 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:51.886303 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:35:52.022875 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:52.022852 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58578f969f-ndgrz"] Jul 22 04:35:52.024817 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:35:52.024792 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68e16db6_39ea_4c19_8125_f69b9a315485.slice/crio-d3b573175fa7c276ded29bd2c49e32f51aad683f7206f8233b8382c1590c6352 WatchSource:0}: Error finding container d3b573175fa7c276ded29bd2c49e32f51aad683f7206f8233b8382c1590c6352: Status 404 returned error can't find the container with id d3b573175fa7c276ded29bd2c49e32f51aad683f7206f8233b8382c1590c6352 Jul 22 04:35:52.627089 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:52.627031 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58578f969f-ndgrz" event={"ID":"68e16db6-39ea-4c19-8125-f69b9a315485","Type":"ContainerStarted","Data":"5c0c6ce520d01ff18f69a27c08e102cda0361beaf7bcfe5bfa373670404f2f7a"} Jul 22 04:35:52.627474 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:52.627112 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58578f969f-ndgrz" event={"ID":"68e16db6-39ea-4c19-8125-f69b9a315485","Type":"ContainerStarted","Data":"d3b573175fa7c276ded29bd2c49e32f51aad683f7206f8233b8382c1590c6352"} Jul 22 04:35:52.645926 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:35:52.645877 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58578f969f-ndgrz" podStartSLOduration=1.6458627689999998 podStartE2EDuration="1.645862769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:35:52.642842022 +0000 UTC m=+676.732121222" watchObservedRunningTime="2026-07-22 04:35:52.645862769 +0000 UTC m=+676.735141981" Jul 22 04:36:01.887393 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:01.887346 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:36:01.887889 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:01.887449 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:36:01.892319 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:01.892293 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:36:02.684541 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:02.684510 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58578f969f-ndgrz" Jul 22 04:36:02.735922 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:02.735890 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d7d79d55-gwgzb"] Jul 22 04:36:18.761635 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.761603 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf"] Jul 22 04:36:18.766245 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.766229 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.768583 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.768552 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vrt4d\"" Jul 22 04:36:18.768583 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.768573 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 04:36:18.769248 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.769231 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 22 04:36:18.769307 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.769244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 22 04:36:18.769307 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.769249 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 04:36:18.773357 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.773335 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf"] Jul 22 04:36:18.887706 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.887669 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9036e68c-5f88-4940-85e4-8eac981087fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.887906 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.887782 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljbvh\" (UniqueName: \"kubernetes.io/projected/9036e68c-5f88-4940-85e4-8eac981087fd-kube-api-access-ljbvh\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.887906 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.887813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9036e68c-5f88-4940-85e4-8eac981087fd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.989160 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.989127 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9036e68c-5f88-4940-85e4-8eac981087fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.989368 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.989220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljbvh\" (UniqueName: \"kubernetes.io/projected/9036e68c-5f88-4940-85e4-8eac981087fd-kube-api-access-ljbvh\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.989368 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.989269 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9036e68c-5f88-4940-85e4-8eac981087fd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.989368 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:36:18.989279 2566 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 22 04:36:18.989368 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:36:18.989351 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9036e68c-5f88-4940-85e4-8eac981087fd-plugin-serving-cert podName:9036e68c-5f88-4940-85e4-8eac981087fd nodeName:}" failed. No retries permitted until 2026-07-22 04:36:19.489330687 +0000 UTC m=+703.578609858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/9036e68c-5f88-4940-85e4-8eac981087fd-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-lf7nf" (UID: "9036e68c-5f88-4940-85e4-8eac981087fd") : secret "plugin-serving-cert" not found Jul 22 04:36:18.990161 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.990135 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9036e68c-5f88-4940-85e4-8eac981087fd-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:18.999732 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:18.999700 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljbvh\" (UniqueName: \"kubernetes.io/projected/9036e68c-5f88-4940-85e4-8eac981087fd-kube-api-access-ljbvh\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:19.495138 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:19.495099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9036e68c-5f88-4940-85e4-8eac981087fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:19.497644 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:19.497610 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9036e68c-5f88-4940-85e4-8eac981087fd-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-lf7nf\" (UID: \"9036e68c-5f88-4940-85e4-8eac981087fd\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:19.679092 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:19.679034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" Jul 22 04:36:19.816688 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:19.816663 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf"] Jul 22 04:36:19.818804 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:36:19.818773 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9036e68c_5f88_4940_85e4_8eac981087fd.slice/crio-817f02aefa7b69a07f29f2857b9bd52868e66ee5ab3b21981a498b84c58b3b79 WatchSource:0}: Error finding container 817f02aefa7b69a07f29f2857b9bd52868e66ee5ab3b21981a498b84c58b3b79: Status 404 returned error can't find the container with id 817f02aefa7b69a07f29f2857b9bd52868e66ee5ab3b21981a498b84c58b3b79 Jul 22 04:36:20.780704 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:20.780660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" event={"ID":"9036e68c-5f88-4940-85e4-8eac981087fd","Type":"ContainerStarted","Data":"817f02aefa7b69a07f29f2857b9bd52868e66ee5ab3b21981a498b84c58b3b79"} Jul 22 04:36:27.762391 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:27.762266 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56d7d79d55-gwgzb" podUID="1c513081-93dd-4c37-a8bd-797233855254" containerName="console" containerID="cri-o://a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82" gracePeriod=15 Jul 22 04:36:28.548591 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:28.548551 2566 patch_prober.go:28] interesting pod/console-56d7d79d55-gwgzb container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" start-of-body= Jul 22 04:36:28.548767 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:28.548610 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-56d7d79d55-gwgzb" podUID="1c513081-93dd-4c37-a8bd-797233855254" containerName="console" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" Jul 22 04:36:29.250373 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.250311 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7"] Jul 22 04:36:29.276975 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.276944 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7"] Jul 22 04:36:29.277219 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.277090 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.279812 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.279782 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bjcws\"" Jul 22 04:36:29.390242 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.390206 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrwqh\" (UniqueName: \"kubernetes.io/projected/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-kube-api-access-zrwqh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8wjh7\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.390424 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.390327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8wjh7\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.491580 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.491540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrwqh\" (UniqueName: \"kubernetes.io/projected/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-kube-api-access-zrwqh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8wjh7\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.491774 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.491659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8wjh7\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.492050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.492030 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8wjh7\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.501659 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.501583 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrwqh\" (UniqueName: \"kubernetes.io/projected/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-kube-api-access-zrwqh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-8wjh7\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:29.591006 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:29.590961 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:38.548423 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:38.548380 2566 patch_prober.go:28] interesting pod/console-56d7d79d55-gwgzb container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" start-of-body= Jul 22 04:36:38.548902 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:38.548457 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-56d7d79d55-gwgzb" podUID="1c513081-93dd-4c37-a8bd-797233855254" containerName="console" probeResult="failure" output="Get \"https://10.132.0.26:8443/health\": dial tcp 10.132.0.26:8443: connect: connection refused" Jul 22 04:36:40.571600 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.571569 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:36:40.575652 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.575629 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7"] Jul 22 04:36:40.576627 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:36:40.576602 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6fd1b75_43a1_43c4_bffd_7d4267e2a7ba.slice/crio-899deb1671a77144b241580b7e0056227f20239e9048c8c45ac46fe79c445614 WatchSource:0}: Error finding container 899deb1671a77144b241580b7e0056227f20239e9048c8c45ac46fe79c445614: Status 404 returned error can't find the container with id 899deb1671a77144b241580b7e0056227f20239e9048c8c45ac46fe79c445614 Jul 22 04:36:40.695272 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695182 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-trusted-ca-bundle\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695272 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695233 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-serving-cert\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695272 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695260 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glpnt\" (UniqueName: \"kubernetes.io/projected/1c513081-93dd-4c37-a8bd-797233855254-kube-api-access-glpnt\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695569 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695286 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-service-ca\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695569 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695476 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-oauth-serving-cert\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695569 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695511 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-oauth-config\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695569 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695565 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-console-config\") pod \"1c513081-93dd-4c37-a8bd-797233855254\" (UID: \"1c513081-93dd-4c37-a8bd-797233855254\") " Jul 22 04:36:40.695776 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695678 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:36:40.695829 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695699 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:36:40.695885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695851 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:36:40.695885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695864 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.695885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.695883 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-trusted-ca-bundle\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.696143 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.696122 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-console-config" (OuterVolumeSpecName: "console-config") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:36:40.697494 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.697475 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:36:40.697567 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.697524 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c513081-93dd-4c37-a8bd-797233855254-kube-api-access-glpnt" (OuterVolumeSpecName: "kube-api-access-glpnt") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "kube-api-access-glpnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:36:40.697567 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.697526 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c513081-93dd-4c37-a8bd-797233855254" (UID: "1c513081-93dd-4c37-a8bd-797233855254"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:36:40.797275 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.797246 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-oauth-serving-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.797275 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.797271 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-oauth-config\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.797275 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.797280 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c513081-93dd-4c37-a8bd-797233855254-console-config\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.797521 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.797294 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c513081-93dd-4c37-a8bd-797233855254-console-serving-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.797521 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.797306 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glpnt\" (UniqueName: \"kubernetes.io/projected/1c513081-93dd-4c37-a8bd-797233855254-kube-api-access-glpnt\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:36:40.916670 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.916635 2566 generic.go:356] "Generic (PLEG): container finished" podID="1c513081-93dd-4c37-a8bd-797233855254" containerID="a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82" exitCode=0 Jul 22 04:36:40.916857 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.916684 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d7d79d55-gwgzb" Jul 22 04:36:40.916857 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.916721 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d7d79d55-gwgzb" event={"ID":"1c513081-93dd-4c37-a8bd-797233855254","Type":"ContainerDied","Data":"a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82"} Jul 22 04:36:40.916857 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.916771 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d7d79d55-gwgzb" event={"ID":"1c513081-93dd-4c37-a8bd-797233855254","Type":"ContainerDied","Data":"8f02f4c31cdc0d2212de48541899ced07afda9447aa98340cfb3cac0c9460433"} Jul 22 04:36:40.916857 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.916799 2566 scope.go:117] "RemoveContainer" containerID="a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82" Jul 22 04:36:40.918396 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.918371 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" event={"ID":"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba","Type":"ContainerStarted","Data":"899deb1671a77144b241580b7e0056227f20239e9048c8c45ac46fe79c445614"} Jul 22 04:36:40.920113 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.920087 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" event={"ID":"9036e68c-5f88-4940-85e4-8eac981087fd","Type":"ContainerStarted","Data":"6b343f9f1c1a4c7ab3766c40c0026a73c138681dbafb7af9e63a1ea8db9d7532"} Jul 22 04:36:40.926929 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.926912 2566 scope.go:117] "RemoveContainer" containerID="a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82" Jul 22 04:36:40.927235 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:36:40.927216 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82\": container with ID starting with a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82 not found: ID does not exist" containerID="a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82" Jul 22 04:36:40.927287 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.927242 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82"} err="failed to get container status \"a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82\": rpc error: code = NotFound desc = could not find container \"a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82\": container with ID starting with a7a5d3f86f1ebacacc5a55cf2430148492b1eb72330dd7b194f8f1aa3d334d82 not found: ID does not exist" Jul 22 04:36:40.937499 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.937459 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-lf7nf" podStartSLOduration=2.259651141 podStartE2EDuration="22.937445796s" totalImagesPullingTime="20.677794655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:36:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:36:40.935658191 +0000 UTC m=+725.024937364" watchObservedRunningTime="2026-07-22 04:36:40.937445796 +0000 UTC m=+725.026724968" Jul 22 04:36:40.955257 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.955184 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d7d79d55-gwgzb"] Jul 22 04:36:40.961064 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:40.961037 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56d7d79d55-gwgzb"] Jul 22 04:36:42.938040 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:42.938005 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c513081-93dd-4c37-a8bd-797233855254" path="/var/lib/kubelet/pods/1c513081-93dd-4c37-a8bd-797233855254/volumes" Jul 22 04:36:44.949638 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:44.949596 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" event={"ID":"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba","Type":"ContainerStarted","Data":"91eccbb0a03a6e91aedebc0cd8622191e464915adcb7ec0a46e695c6f975a4c8"} Jul 22 04:36:44.950050 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:44.949659 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:36:44.968460 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:44.968129 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" podStartSLOduration=11.768109604 podStartE2EDuration="15.968112341s" totalImagesPullingTime="4.200002737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:36:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:36:44.966339369 +0000 UTC m=+729.055618540" watchObservedRunningTime="2026-07-22 04:36:44.968112341 +0000 UTC m=+729.057391514" Jul 22 04:36:55.957506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:36:55.957475 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:37:07.854890 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:07.854855 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7"] Jul 22 04:37:07.855391 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:07.855056 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" podUID="a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" containerName="manager" containerID="cri-o://91eccbb0a03a6e91aedebc0cd8622191e464915adcb7ec0a46e695c6f975a4c8" gracePeriod=10 Jul 22 04:37:08.073562 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.073531 2566 generic.go:356] "Generic (PLEG): container finished" podID="a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" containerID="91eccbb0a03a6e91aedebc0cd8622191e464915adcb7ec0a46e695c6f975a4c8" exitCode=0 Jul 22 04:37:08.073699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.073603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" event={"ID":"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba","Type":"ContainerDied","Data":"91eccbb0a03a6e91aedebc0cd8622191e464915adcb7ec0a46e695c6f975a4c8"} Jul 22 04:37:08.665733 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.665700 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:37:08.728063 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.728030 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrwqh\" (UniqueName: \"kubernetes.io/projected/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-kube-api-access-zrwqh\") pod \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " Jul 22 04:37:08.728257 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.728083 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-extensions-socket-volume\") pod \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\" (UID: \"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba\") " Jul 22 04:37:08.728608 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.728584 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" (UID: "a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:37:08.730270 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.730244 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-kube-api-access-zrwqh" (OuterVolumeSpecName: "kube-api-access-zrwqh") pod "a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" (UID: "a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba"). InnerVolumeSpecName "kube-api-access-zrwqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:37:08.828742 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.828653 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrwqh\" (UniqueName: \"kubernetes.io/projected/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-kube-api-access-zrwqh\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:08.828742 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:08.828690 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba-extensions-socket-volume\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:09.080552 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:09.080471 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" event={"ID":"a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba","Type":"ContainerDied","Data":"899deb1671a77144b241580b7e0056227f20239e9048c8c45ac46fe79c445614"} Jul 22 04:37:09.080552 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:09.080498 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7" Jul 22 04:37:09.080552 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:09.080535 2566 scope.go:117] "RemoveContainer" containerID="91eccbb0a03a6e91aedebc0cd8622191e464915adcb7ec0a46e695c6f975a4c8" Jul 22 04:37:09.098879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:09.098853 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7"] Jul 22 04:37:09.102252 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:09.102226 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-8wjh7"] Jul 22 04:37:10.931206 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:10.931172 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" path="/var/lib/kubelet/pods/a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba/volumes" Jul 22 04:37:16.939484 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939449 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:16.939863 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939854 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c513081-93dd-4c37-a8bd-797233855254" containerName="console" Jul 22 04:37:16.939904 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939867 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c513081-93dd-4c37-a8bd-797233855254" containerName="console" Jul 22 04:37:16.939904 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939884 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" containerName="manager" Jul 22 04:37:16.939904 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939889 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" containerName="manager" Jul 22 04:37:16.939994 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939955 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c513081-93dd-4c37-a8bd-797233855254" containerName="console" Jul 22 04:37:16.939994 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.939962 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6fd1b75-43a1-43c4-bffd-7d4267e2a7ba" containerName="manager" Jul 22 04:37:16.942647 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.942630 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:16.944840 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.944821 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 04:37:16.950656 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.950635 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:16.998130 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.998059 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac5e2002-9ad2-41dd-84e7-8f9675af2249-config-file\") pod \"limitador-limitador-64c94fd64f-4jjzm\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:16.998310 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:16.998179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86wqn\" (UniqueName: \"kubernetes.io/projected/ac5e2002-9ad2-41dd-84e7-8f9675af2249-kube-api-access-86wqn\") pod \"limitador-limitador-64c94fd64f-4jjzm\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:17.023935 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.023902 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:17.098967 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.098925 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac5e2002-9ad2-41dd-84e7-8f9675af2249-config-file\") pod \"limitador-limitador-64c94fd64f-4jjzm\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:17.099151 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.098983 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86wqn\" (UniqueName: \"kubernetes.io/projected/ac5e2002-9ad2-41dd-84e7-8f9675af2249-kube-api-access-86wqn\") pod \"limitador-limitador-64c94fd64f-4jjzm\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:17.099564 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.099544 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac5e2002-9ad2-41dd-84e7-8f9675af2249-config-file\") pod \"limitador-limitador-64c94fd64f-4jjzm\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:17.106709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.106679 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-86wqn\" (UniqueName: \"kubernetes.io/projected/ac5e2002-9ad2-41dd-84e7-8f9675af2249-kube-api-access-86wqn\") pod \"limitador-limitador-64c94fd64f-4jjzm\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:17.253148 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.253046 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:17.388692 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.388667 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:17.391172 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:37:17.391146 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5e2002_9ad2_41dd_84e7_8f9675af2249.slice/crio-235e7c3933536268e088fd04c0868ca0f1dcac6ec128823a6198ec8401f0185f WatchSource:0}: Error finding container 235e7c3933536268e088fd04c0868ca0f1dcac6ec128823a6198ec8401f0185f: Status 404 returned error can't find the container with id 235e7c3933536268e088fd04c0868ca0f1dcac6ec128823a6198ec8401f0185f Jul 22 04:37:17.863367 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.863335 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-k6psj"] Jul 22 04:37:17.866393 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.866373 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:17.868621 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.868597 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c6jjg\"" Jul 22 04:37:17.873264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.873240 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k6psj"] Jul 22 04:37:17.905997 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:17.905966 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8jb9\" (UniqueName: \"kubernetes.io/projected/b428af04-fb1b-4582-b277-0670d5bf2b97-kube-api-access-n8jb9\") pod \"authorino-75d556bd77-k6psj\" (UID: \"b428af04-fb1b-4582-b277-0670d5bf2b97\") " pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:18.006709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:18.006677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8jb9\" (UniqueName: \"kubernetes.io/projected/b428af04-fb1b-4582-b277-0670d5bf2b97-kube-api-access-n8jb9\") pod \"authorino-75d556bd77-k6psj\" (UID: \"b428af04-fb1b-4582-b277-0670d5bf2b97\") " pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:18.016866 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:18.016838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8jb9\" (UniqueName: \"kubernetes.io/projected/b428af04-fb1b-4582-b277-0670d5bf2b97-kube-api-access-n8jb9\") pod \"authorino-75d556bd77-k6psj\" (UID: \"b428af04-fb1b-4582-b277-0670d5bf2b97\") " pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:18.131747 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:18.131645 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" event={"ID":"ac5e2002-9ad2-41dd-84e7-8f9675af2249","Type":"ContainerStarted","Data":"235e7c3933536268e088fd04c0868ca0f1dcac6ec128823a6198ec8401f0185f"} Jul 22 04:37:18.176232 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:18.176195 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:18.363711 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:18.363674 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k6psj"] Jul 22 04:37:18.368867 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:37:18.368800 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb428af04_fb1b_4582_b277_0670d5bf2b97.slice/crio-1ea0b1cd3cf73281176385f3a84f1758283bb7a4bd51fd137b89a1f5f109f64d WatchSource:0}: Error finding container 1ea0b1cd3cf73281176385f3a84f1758283bb7a4bd51fd137b89a1f5f109f64d: Status 404 returned error can't find the container with id 1ea0b1cd3cf73281176385f3a84f1758283bb7a4bd51fd137b89a1f5f109f64d Jul 22 04:37:19.140493 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:19.140453 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k6psj" event={"ID":"b428af04-fb1b-4582-b277-0670d5bf2b97","Type":"ContainerStarted","Data":"1ea0b1cd3cf73281176385f3a84f1758283bb7a4bd51fd137b89a1f5f109f64d"} Jul 22 04:37:23.163519 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:23.163485 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k6psj" event={"ID":"b428af04-fb1b-4582-b277-0670d5bf2b97","Type":"ContainerStarted","Data":"a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3"} Jul 22 04:37:23.165039 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:23.165014 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" event={"ID":"ac5e2002-9ad2-41dd-84e7-8f9675af2249","Type":"ContainerStarted","Data":"bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2"} Jul 22 04:37:23.165173 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:23.165093 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:23.179890 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:23.179842 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-k6psj" podStartSLOduration=2.423944431 podStartE2EDuration="6.179820102s" totalImagesPullingTime="3.755875671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:23.176901906 +0000 UTC m=+767.266181078" watchObservedRunningTime="2026-07-22 04:37:23.179820102 +0000 UTC m=+767.269099273" Jul 22 04:37:23.193504 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:23.193447 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" podStartSLOduration=2.460464752 podStartE2EDuration="7.193428135s" totalImagesPullingTime="4.732963383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:23.191695626 +0000 UTC m=+767.280974797" watchObservedRunningTime="2026-07-22 04:37:23.193428135 +0000 UTC m=+767.282707308" Jul 22 04:37:31.880459 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:31.880421 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:31.880982 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:31.880643 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" podUID="ac5e2002-9ad2-41dd-84e7-8f9675af2249" containerName="limitador" containerID="cri-o://bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2" gracePeriod=30 Jul 22 04:37:31.881431 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:31.881411 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:32.437526 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.437502 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:32.544111 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.543980 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac5e2002-9ad2-41dd-84e7-8f9675af2249-config-file\") pod \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " Jul 22 04:37:32.544111 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.544090 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-86wqn\" (UniqueName: \"kubernetes.io/projected/ac5e2002-9ad2-41dd-84e7-8f9675af2249-kube-api-access-86wqn\") pod \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\" (UID: \"ac5e2002-9ad2-41dd-84e7-8f9675af2249\") " Jul 22 04:37:32.544354 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.544329 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac5e2002-9ad2-41dd-84e7-8f9675af2249-config-file" (OuterVolumeSpecName: "config-file") pod "ac5e2002-9ad2-41dd-84e7-8f9675af2249" (UID: "ac5e2002-9ad2-41dd-84e7-8f9675af2249"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:37:32.546143 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.546119 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac5e2002-9ad2-41dd-84e7-8f9675af2249-kube-api-access-86wqn" (OuterVolumeSpecName: "kube-api-access-86wqn") pod "ac5e2002-9ad2-41dd-84e7-8f9675af2249" (UID: "ac5e2002-9ad2-41dd-84e7-8f9675af2249"). InnerVolumeSpecName "kube-api-access-86wqn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:37:32.645633 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.645594 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-86wqn\" (UniqueName: \"kubernetes.io/projected/ac5e2002-9ad2-41dd-84e7-8f9675af2249-kube-api-access-86wqn\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:32.645633 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:32.645627 2566 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/ac5e2002-9ad2-41dd-84e7-8f9675af2249-config-file\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:33.220122 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.220015 2566 generic.go:356] "Generic (PLEG): container finished" podID="ac5e2002-9ad2-41dd-84e7-8f9675af2249" containerID="bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2" exitCode=0 Jul 22 04:37:33.220122 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.220090 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" Jul 22 04:37:33.220122 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.220089 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" event={"ID":"ac5e2002-9ad2-41dd-84e7-8f9675af2249","Type":"ContainerDied","Data":"bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2"} Jul 22 04:37:33.220585 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.220146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4jjzm" event={"ID":"ac5e2002-9ad2-41dd-84e7-8f9675af2249","Type":"ContainerDied","Data":"235e7c3933536268e088fd04c0868ca0f1dcac6ec128823a6198ec8401f0185f"} Jul 22 04:37:33.220585 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.220170 2566 scope.go:117] "RemoveContainer" containerID="bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2" Jul 22 04:37:33.229000 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.228980 2566 scope.go:117] "RemoveContainer" containerID="bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2" Jul 22 04:37:33.229273 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:37:33.229251 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2\": container with ID starting with bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2 not found: ID does not exist" containerID="bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2" Jul 22 04:37:33.229340 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.229284 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2"} err="failed to get container status \"bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2\": rpc error: code = NotFound desc = could not find container \"bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2\": container with ID starting with bccb32a1ea1807c00cc0bf2522e0a4bb8db4a688d66dae5f8002a8e5463629a2 not found: ID does not exist" Jul 22 04:37:33.235810 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.235789 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:33.239483 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:33.239464 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4jjzm"] Jul 22 04:37:34.931517 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:34.931485 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac5e2002-9ad2-41dd-84e7-8f9675af2249" path="/var/lib/kubelet/pods/ac5e2002-9ad2-41dd-84e7-8f9675af2249/volumes" Jul 22 04:37:45.546586 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.546547 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g29s9"] Jul 22 04:37:45.546954 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.546926 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac5e2002-9ad2-41dd-84e7-8f9675af2249" containerName="limitador" Jul 22 04:37:45.546954 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.546948 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac5e2002-9ad2-41dd-84e7-8f9675af2249" containerName="limitador" Jul 22 04:37:45.547029 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.547002 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac5e2002-9ad2-41dd-84e7-8f9675af2249" containerName="limitador" Jul 22 04:37:45.551287 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.551265 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:45.557689 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.557662 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g29s9"] Jul 22 04:37:45.653891 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.653856 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v826n\" (UniqueName: \"kubernetes.io/projected/40f0e49c-b5eb-4df3-88f4-d62584e4d662-kube-api-access-v826n\") pod \"authorino-5d4b67c446-g29s9\" (UID: \"40f0e49c-b5eb-4df3-88f4-d62584e4d662\") " pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:45.755242 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.755206 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v826n\" (UniqueName: \"kubernetes.io/projected/40f0e49c-b5eb-4df3-88f4-d62584e4d662-kube-api-access-v826n\") pod \"authorino-5d4b67c446-g29s9\" (UID: \"40f0e49c-b5eb-4df3-88f4-d62584e4d662\") " pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:45.762651 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.762620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v826n\" (UniqueName: \"kubernetes.io/projected/40f0e49c-b5eb-4df3-88f4-d62584e4d662-kube-api-access-v826n\") pod \"authorino-5d4b67c446-g29s9\" (UID: \"40f0e49c-b5eb-4df3-88f4-d62584e4d662\") " pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:45.768316 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.768295 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g29s9"] Jul 22 04:37:45.768504 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.768493 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:45.792264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.792237 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6594765d57-pz6vh"] Jul 22 04:37:45.795851 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.795832 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:45.802617 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.802559 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6594765d57-pz6vh"] Jul 22 04:37:45.856615 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.856590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm926\" (UniqueName: \"kubernetes.io/projected/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4-kube-api-access-pm926\") pod \"authorino-6594765d57-pz6vh\" (UID: \"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4\") " pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:45.899374 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.899350 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g29s9"] Jul 22 04:37:45.901574 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:37:45.901548 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod40f0e49c_b5eb_4df3_88f4_d62584e4d662.slice/crio-06401c5bad006d16a446fb1c904d765f7f3fa820a86f5a9891fd11def3b78062 WatchSource:0}: Error finding container 06401c5bad006d16a446fb1c904d765f7f3fa820a86f5a9891fd11def3b78062: Status 404 returned error can't find the container with id 06401c5bad006d16a446fb1c904d765f7f3fa820a86f5a9891fd11def3b78062 Jul 22 04:37:45.957276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.957249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm926\" (UniqueName: \"kubernetes.io/projected/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4-kube-api-access-pm926\") pod \"authorino-6594765d57-pz6vh\" (UID: \"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4\") " pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:45.968833 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:45.966594 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm926\" (UniqueName: \"kubernetes.io/projected/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4-kube-api-access-pm926\") pod \"authorino-6594765d57-pz6vh\" (UID: \"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4\") " pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:46.018726 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.018692 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6594765d57-pz6vh"] Jul 22 04:37:46.018954 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.018930 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:46.050114 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.050057 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-659977b69c-94chr"] Jul 22 04:37:46.053707 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.053661 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.055756 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.055733 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 22 04:37:46.061336 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.060383 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-659977b69c-94chr"] Jul 22 04:37:46.149215 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.149187 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6594765d57-pz6vh"] Jul 22 04:37:46.151161 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:37:46.151133 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbb6a66dd_e54a_4974_b1fd_bd15973c3ae4.slice/crio-083f877c116a95b64ea6fc3fd188473ea3514b974adb7d36ace1253cf0d7a8b7 WatchSource:0}: Error finding container 083f877c116a95b64ea6fc3fd188473ea3514b974adb7d36ace1253cf0d7a8b7: Status 404 returned error can't find the container with id 083f877c116a95b64ea6fc3fd188473ea3514b974adb7d36ace1253cf0d7a8b7 Jul 22 04:37:46.158566 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.158540 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28e02621-336a-4b7b-a6a0-633750c922c7-tls-cert\") pod \"authorino-659977b69c-94chr\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.158655 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.158591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d5kw\" (UniqueName: \"kubernetes.io/projected/28e02621-336a-4b7b-a6a0-633750c922c7-kube-api-access-2d5kw\") pod \"authorino-659977b69c-94chr\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.259263 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.259239 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28e02621-336a-4b7b-a6a0-633750c922c7-tls-cert\") pod \"authorino-659977b69c-94chr\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.259360 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.259277 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d5kw\" (UniqueName: \"kubernetes.io/projected/28e02621-336a-4b7b-a6a0-633750c922c7-kube-api-access-2d5kw\") pod \"authorino-659977b69c-94chr\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.261438 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.261420 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28e02621-336a-4b7b-a6a0-633750c922c7-tls-cert\") pod \"authorino-659977b69c-94chr\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.267194 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.267171 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d5kw\" (UniqueName: \"kubernetes.io/projected/28e02621-336a-4b7b-a6a0-633750c922c7-kube-api-access-2d5kw\") pod \"authorino-659977b69c-94chr\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.296731 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.296703 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6594765d57-pz6vh" event={"ID":"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4","Type":"ContainerStarted","Data":"083f877c116a95b64ea6fc3fd188473ea3514b974adb7d36ace1253cf0d7a8b7"} Jul 22 04:37:46.298291 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.298268 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g29s9" event={"ID":"40f0e49c-b5eb-4df3-88f4-d62584e4d662","Type":"ContainerStarted","Data":"06401c5bad006d16a446fb1c904d765f7f3fa820a86f5a9891fd11def3b78062"} Jul 22 04:37:46.365135 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.365035 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:37:46.498926 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:46.498902 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-659977b69c-94chr"] Jul 22 04:37:46.501164 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:37:46.501136 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28e02621_336a_4b7b_a6a0_633750c922c7.slice/crio-31ec16df72c472e978164c87b2d3de41c1682e609f35f1465b69b4cd8b3ed8fb WatchSource:0}: Error finding container 31ec16df72c472e978164c87b2d3de41c1682e609f35f1465b69b4cd8b3ed8fb: Status 404 returned error can't find the container with id 31ec16df72c472e978164c87b2d3de41c1682e609f35f1465b69b4cd8b3ed8fb Jul 22 04:37:47.304737 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.304640 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659977b69c-94chr" event={"ID":"28e02621-336a-4b7b-a6a0-633750c922c7","Type":"ContainerStarted","Data":"8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7"} Jul 22 04:37:47.304737 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.304681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659977b69c-94chr" event={"ID":"28e02621-336a-4b7b-a6a0-633750c922c7","Type":"ContainerStarted","Data":"31ec16df72c472e978164c87b2d3de41c1682e609f35f1465b69b4cd8b3ed8fb"} Jul 22 04:37:47.306252 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.306224 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6594765d57-pz6vh" podUID="bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" containerName="authorino" containerID="cri-o://66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32" gracePeriod=30 Jul 22 04:37:47.306252 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.306239 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6594765d57-pz6vh" event={"ID":"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4","Type":"ContainerStarted","Data":"66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32"} Jul 22 04:37:47.307782 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.307753 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g29s9" event={"ID":"40f0e49c-b5eb-4df3-88f4-d62584e4d662","Type":"ContainerStarted","Data":"f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016"} Jul 22 04:37:47.307921 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.307809 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-g29s9" podUID="40f0e49c-b5eb-4df3-88f4-d62584e4d662" containerName="authorino" containerID="cri-o://f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016" gracePeriod=30 Jul 22 04:37:47.322182 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.322132 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-659977b69c-94chr" podStartSLOduration=0.929491688 podStartE2EDuration="1.32211524s" totalImagesPullingTime="392.623552ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:47.319222002 +0000 UTC m=+791.408501175" watchObservedRunningTime="2026-07-22 04:37:47.32211524 +0000 UTC m=+791.411394412" Jul 22 04:37:47.333216 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.333003 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-g29s9" podStartSLOduration=2.001205473 podStartE2EDuration="2.33298717s" totalImagesPullingTime="331.781697ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:47.332447522 +0000 UTC m=+791.421726695" watchObservedRunningTime="2026-07-22 04:37:47.33298717 +0000 UTC m=+791.422266342" Jul 22 04:37:47.345129 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.345101 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k6psj"] Jul 22 04:37:47.345275 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.345255 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-k6psj" podUID="b428af04-fb1b-4582-b277-0670d5bf2b97" containerName="authorino" containerID="cri-o://a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3" gracePeriod=30 Jul 22 04:37:47.349520 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.349478 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6594765d57-pz6vh" podStartSLOduration=2.057447256 podStartE2EDuration="2.349464277s" totalImagesPullingTime="292.017021ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:47.34877269 +0000 UTC m=+791.438051862" watchObservedRunningTime="2026-07-22 04:37:47.349464277 +0000 UTC m=+791.438743448" Jul 22 04:37:47.630947 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.630804 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:47.637923 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.637904 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:47.641057 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.641040 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:47.671695 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.671665 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v826n\" (UniqueName: \"kubernetes.io/projected/40f0e49c-b5eb-4df3-88f4-d62584e4d662-kube-api-access-v826n\") pod \"40f0e49c-b5eb-4df3-88f4-d62584e4d662\" (UID: \"40f0e49c-b5eb-4df3-88f4-d62584e4d662\") " Jul 22 04:37:47.671947 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.671931 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pm926\" (UniqueName: \"kubernetes.io/projected/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4-kube-api-access-pm926\") pod \"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4\" (UID: \"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4\") " Jul 22 04:37:47.672110 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.672095 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8jb9\" (UniqueName: \"kubernetes.io/projected/b428af04-fb1b-4582-b277-0670d5bf2b97-kube-api-access-n8jb9\") pod \"b428af04-fb1b-4582-b277-0670d5bf2b97\" (UID: \"b428af04-fb1b-4582-b277-0670d5bf2b97\") " Jul 22 04:37:47.674450 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.674419 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40f0e49c-b5eb-4df3-88f4-d62584e4d662-kube-api-access-v826n" (OuterVolumeSpecName: "kube-api-access-v826n") pod "40f0e49c-b5eb-4df3-88f4-d62584e4d662" (UID: "40f0e49c-b5eb-4df3-88f4-d62584e4d662"). InnerVolumeSpecName "kube-api-access-v826n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:37:47.674761 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.674731 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b428af04-fb1b-4582-b277-0670d5bf2b97-kube-api-access-n8jb9" (OuterVolumeSpecName: "kube-api-access-n8jb9") pod "b428af04-fb1b-4582-b277-0670d5bf2b97" (UID: "b428af04-fb1b-4582-b277-0670d5bf2b97"). InnerVolumeSpecName "kube-api-access-n8jb9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:37:47.674955 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.674926 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4-kube-api-access-pm926" (OuterVolumeSpecName: "kube-api-access-pm926") pod "bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" (UID: "bb6a66dd-e54a-4974-b1fd-bd15973c3ae4"). InnerVolumeSpecName "kube-api-access-pm926". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:37:47.773579 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.773555 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v826n\" (UniqueName: \"kubernetes.io/projected/40f0e49c-b5eb-4df3-88f4-d62584e4d662-kube-api-access-v826n\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:47.773579 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.773581 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pm926\" (UniqueName: \"kubernetes.io/projected/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4-kube-api-access-pm926\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:47.773720 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:47.773592 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8jb9\" (UniqueName: \"kubernetes.io/projected/b428af04-fb1b-4582-b277-0670d5bf2b97-kube-api-access-n8jb9\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:37:48.315451 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.315404 2566 generic.go:356] "Generic (PLEG): container finished" podID="40f0e49c-b5eb-4df3-88f4-d62584e4d662" containerID="f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016" exitCode=0 Jul 22 04:37:48.315451 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.315439 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g29s9" event={"ID":"40f0e49c-b5eb-4df3-88f4-d62584e4d662","Type":"ContainerDied","Data":"f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016"} Jul 22 04:37:48.315962 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.315477 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g29s9" event={"ID":"40f0e49c-b5eb-4df3-88f4-d62584e4d662","Type":"ContainerDied","Data":"06401c5bad006d16a446fb1c904d765f7f3fa820a86f5a9891fd11def3b78062"} Jul 22 04:37:48.315962 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.315421 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g29s9" Jul 22 04:37:48.315962 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.315502 2566 scope.go:117] "RemoveContainer" containerID="f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016" Jul 22 04:37:48.317116 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.317043 2566 generic.go:356] "Generic (PLEG): container finished" podID="b428af04-fb1b-4582-b277-0670d5bf2b97" containerID="a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3" exitCode=0 Jul 22 04:37:48.317217 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.317141 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-k6psj" Jul 22 04:37:48.317285 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.317236 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k6psj" event={"ID":"b428af04-fb1b-4582-b277-0670d5bf2b97","Type":"ContainerDied","Data":"a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3"} Jul 22 04:37:48.317285 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.317272 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-k6psj" event={"ID":"b428af04-fb1b-4582-b277-0670d5bf2b97","Type":"ContainerDied","Data":"1ea0b1cd3cf73281176385f3a84f1758283bb7a4bd51fd137b89a1f5f109f64d"} Jul 22 04:37:48.319088 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.319048 2566 generic.go:356] "Generic (PLEG): container finished" podID="bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" containerID="66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32" exitCode=0 Jul 22 04:37:48.319233 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.319205 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6594765d57-pz6vh" event={"ID":"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4","Type":"ContainerDied","Data":"66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32"} Jul 22 04:37:48.319315 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.319256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6594765d57-pz6vh" event={"ID":"bb6a66dd-e54a-4974-b1fd-bd15973c3ae4","Type":"ContainerDied","Data":"083f877c116a95b64ea6fc3fd188473ea3514b974adb7d36ace1253cf0d7a8b7"} Jul 22 04:37:48.319315 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.319264 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6594765d57-pz6vh" Jul 22 04:37:48.326774 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.326755 2566 scope.go:117] "RemoveContainer" containerID="f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016" Jul 22 04:37:48.327062 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:37:48.327035 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016\": container with ID starting with f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016 not found: ID does not exist" containerID="f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016" Jul 22 04:37:48.327294 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.327265 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016"} err="failed to get container status \"f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016\": rpc error: code = NotFound desc = could not find container \"f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016\": container with ID starting with f4f863d79357b8c0bd9bf80c45e7a89411fa45df3e63fef0b7b879b3b0d7b016 not found: ID does not exist" Jul 22 04:37:48.327388 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.327298 2566 scope.go:117] "RemoveContainer" containerID="a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3" Jul 22 04:37:48.338969 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.338947 2566 scope.go:117] "RemoveContainer" containerID="a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3" Jul 22 04:37:48.339239 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:37:48.339222 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3\": container with ID starting with a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3 not found: ID does not exist" containerID="a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3" Jul 22 04:37:48.339301 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.339245 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3"} err="failed to get container status \"a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3\": rpc error: code = NotFound desc = could not find container \"a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3\": container with ID starting with a002658c8ea18a9049c46fcd8c4a5154d89e2850d05d547cd53c5451609c1ce3 not found: ID does not exist" Jul 22 04:37:48.339301 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.339261 2566 scope.go:117] "RemoveContainer" containerID="66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32" Jul 22 04:37:48.346080 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.346044 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g29s9"] Jul 22 04:37:48.348245 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.348174 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g29s9"] Jul 22 04:37:48.348296 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.348285 2566 scope.go:117] "RemoveContainer" containerID="66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32" Jul 22 04:37:48.348563 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:37:48.348545 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32\": container with ID starting with 66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32 not found: ID does not exist" containerID="66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32" Jul 22 04:37:48.348627 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.348569 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32"} err="failed to get container status \"66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32\": rpc error: code = NotFound desc = could not find container \"66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32\": container with ID starting with 66f57a69f8d283a2d3301cf91677888e6f20f6e247eeeb7e8af6a90d0d091d32 not found: ID does not exist" Jul 22 04:37:48.356396 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.356372 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k6psj"] Jul 22 04:37:48.359581 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.359561 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-k6psj"] Jul 22 04:37:48.369014 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.368992 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6594765d57-pz6vh"] Jul 22 04:37:48.372059 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.372040 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6594765d57-pz6vh"] Jul 22 04:37:48.931549 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.931511 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40f0e49c-b5eb-4df3-88f4-d62584e4d662" path="/var/lib/kubelet/pods/40f0e49c-b5eb-4df3-88f4-d62584e4d662/volumes" Jul 22 04:37:48.931883 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.931869 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b428af04-fb1b-4582-b277-0670d5bf2b97" path="/var/lib/kubelet/pods/b428af04-fb1b-4582-b277-0670d5bf2b97/volumes" Jul 22 04:37:48.932206 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:48.932192 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" path="/var/lib/kubelet/pods/bb6a66dd-e54a-4974-b1fd-bd15973c3ae4/volumes" Jul 22 04:37:51.760507 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760467 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5959cdf964-4ptwj"] Jul 22 04:37:51.760910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760836 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40f0e49c-b5eb-4df3-88f4-d62584e4d662" containerName="authorino" Jul 22 04:37:51.760910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760847 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="40f0e49c-b5eb-4df3-88f4-d62584e4d662" containerName="authorino" Jul 22 04:37:51.760910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760864 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b428af04-fb1b-4582-b277-0670d5bf2b97" containerName="authorino" Jul 22 04:37:51.760910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760870 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b428af04-fb1b-4582-b277-0670d5bf2b97" containerName="authorino" Jul 22 04:37:51.760910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760877 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" containerName="authorino" Jul 22 04:37:51.760910 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760883 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" containerName="authorino" Jul 22 04:37:51.761155 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760942 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b428af04-fb1b-4582-b277-0670d5bf2b97" containerName="authorino" Jul 22 04:37:51.761155 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760952 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="40f0e49c-b5eb-4df3-88f4-d62584e4d662" containerName="authorino" Jul 22 04:37:51.761155 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.760962 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb6a66dd-e54a-4974-b1fd-bd15973c3ae4" containerName="authorino" Jul 22 04:37:51.765430 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.765411 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:51.767773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.767752 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 04:37:51.767879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.767822 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xfh4m\"" Jul 22 04:37:51.767879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.767856 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 04:37:51.778486 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.778457 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5959cdf964-4ptwj"] Jul 22 04:37:51.805765 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.805734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-webhook-cert\") pod \"maas-controller-5959cdf964-4ptwj\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:51.805937 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.805874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tpxz\" (UniqueName: \"kubernetes.io/projected/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-kube-api-access-5tpxz\") pod \"maas-controller-5959cdf964-4ptwj\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:51.906393 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.906359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tpxz\" (UniqueName: \"kubernetes.io/projected/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-kube-api-access-5tpxz\") pod \"maas-controller-5959cdf964-4ptwj\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:51.906609 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.906414 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-webhook-cert\") pod \"maas-controller-5959cdf964-4ptwj\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:51.908880 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.908858 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-webhook-cert\") pod \"maas-controller-5959cdf964-4ptwj\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:51.914294 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:51.914273 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tpxz\" (UniqueName: \"kubernetes.io/projected/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-kube-api-access-5tpxz\") pod \"maas-controller-5959cdf964-4ptwj\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:52.078611 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:52.078526 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:52.218618 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:52.218593 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5959cdf964-4ptwj"] Jul 22 04:37:52.220438 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:37:52.220405 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4dfccf_d5cd_47c3_9dc5_c2938bb1a639.slice/crio-0017c363dd9cadf8b4495bb8848b85efb67012b2d8fd82c622ea95a75513f538 WatchSource:0}: Error finding container 0017c363dd9cadf8b4495bb8848b85efb67012b2d8fd82c622ea95a75513f538: Status 404 returned error can't find the container with id 0017c363dd9cadf8b4495bb8848b85efb67012b2d8fd82c622ea95a75513f538 Jul 22 04:37:52.343752 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:52.343651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5959cdf964-4ptwj" event={"ID":"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639","Type":"ContainerStarted","Data":"0017c363dd9cadf8b4495bb8848b85efb67012b2d8fd82c622ea95a75513f538"} Jul 22 04:37:55.363396 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:55.363354 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5959cdf964-4ptwj" event={"ID":"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639","Type":"ContainerStarted","Data":"2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c"} Jul 22 04:37:55.363940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:55.363489 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:37:55.385628 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:37:55.385573 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5959cdf964-4ptwj" podStartSLOduration=1.784816754 podStartE2EDuration="4.385555935s" totalImagesPullingTime="2.600739181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:37:55.382292414 +0000 UTC m=+799.471571585" watchObservedRunningTime="2026-07-22 04:37:55.385555935 +0000 UTC m=+799.474835106" Jul 22 04:38:06.129537 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.129450 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5959cdf964-4ptwj"] Jul 22 04:38:06.130132 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.129673 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5959cdf964-4ptwj" podUID="6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" containerName="manager" containerID="cri-o://2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c" gracePeriod=30 Jul 22 04:38:06.131539 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.131513 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:38:06.375890 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.375857 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:38:06.423063 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.423025 2566 generic.go:356] "Generic (PLEG): container finished" podID="6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" containerID="2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c" exitCode=0 Jul 22 04:38:06.423244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.423090 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5959cdf964-4ptwj" Jul 22 04:38:06.423244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.423117 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5959cdf964-4ptwj" event={"ID":"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639","Type":"ContainerDied","Data":"2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c"} Jul 22 04:38:06.423244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.423173 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5959cdf964-4ptwj" event={"ID":"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639","Type":"ContainerDied","Data":"0017c363dd9cadf8b4495bb8848b85efb67012b2d8fd82c622ea95a75513f538"} Jul 22 04:38:06.423244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.423195 2566 scope.go:117] "RemoveContainer" containerID="2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c" Jul 22 04:38:06.431886 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.431859 2566 scope.go:117] "RemoveContainer" containerID="2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c" Jul 22 04:38:06.432182 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:38:06.432157 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c\": container with ID starting with 2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c not found: ID does not exist" containerID="2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c" Jul 22 04:38:06.432276 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.432194 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c"} err="failed to get container status \"2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c\": rpc error: code = NotFound desc = could not find container \"2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c\": container with ID starting with 2adf5f8a75b55dfc8f8855f3820d3dc6f6efc240fe98bfa64f640cc8f042306c not found: ID does not exist" Jul 22 04:38:06.436444 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.436426 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5tpxz\" (UniqueName: \"kubernetes.io/projected/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-kube-api-access-5tpxz\") pod \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " Jul 22 04:38:06.436500 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.436493 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-webhook-cert\") pod \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\" (UID: \"6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639\") " Jul 22 04:38:06.438520 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.438499 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-kube-api-access-5tpxz" (OuterVolumeSpecName: "kube-api-access-5tpxz") pod "6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" (UID: "6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639"). InnerVolumeSpecName "kube-api-access-5tpxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:38:06.438686 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.438667 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" (UID: "6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:38:06.537874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.537829 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5tpxz\" (UniqueName: \"kubernetes.io/projected/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-kube-api-access-5tpxz\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:38:06.537874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.537868 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639-webhook-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:38:06.745950 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.745914 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5959cdf964-4ptwj"] Jul 22 04:38:06.752107 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.752057 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5959cdf964-4ptwj"] Jul 22 04:38:06.932236 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:06.932201 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" path="/var/lib/kubelet/pods/6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639/volumes" Jul 22 04:38:25.812710 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.812672 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-96979fff8-m7j7d"] Jul 22 04:38:25.813109 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.813038 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" containerName="manager" Jul 22 04:38:25.813109 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.813048 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" containerName="manager" Jul 22 04:38:25.813191 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.813123 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d4dfccf-d5cd-47c3-9dc5-c2938bb1a639" containerName="manager" Jul 22 04:38:25.817364 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.817346 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:25.819643 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.819623 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 04:38:25.819778 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.819625 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 04:38:25.819778 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.819625 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-tcl88\"" Jul 22 04:38:25.820305 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.820288 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 22 04:38:25.841061 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.841033 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-96979fff8-m7j7d"] Jul 22 04:38:25.909905 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.909871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2311a276-c41e-4079-b677-bc3e45864ebc-maas-api-tls\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:25.910186 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.909975 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2311a276-c41e-4079-b677-bc3e45864ebc-openshift-service-ca\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:25.910186 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:25.910000 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfmm9\" (UniqueName: \"kubernetes.io/projected/2311a276-c41e-4079-b677-bc3e45864ebc-kube-api-access-sfmm9\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.011405 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.011364 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2311a276-c41e-4079-b677-bc3e45864ebc-maas-api-tls\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.011705 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.011478 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2311a276-c41e-4079-b677-bc3e45864ebc-openshift-service-ca\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.011705 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.011503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfmm9\" (UniqueName: \"kubernetes.io/projected/2311a276-c41e-4079-b677-bc3e45864ebc-kube-api-access-sfmm9\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.012160 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.012141 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2311a276-c41e-4079-b677-bc3e45864ebc-openshift-service-ca\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.013832 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.013815 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2311a276-c41e-4079-b677-bc3e45864ebc-maas-api-tls\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.018905 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.018883 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfmm9\" (UniqueName: \"kubernetes.io/projected/2311a276-c41e-4079-b677-bc3e45864ebc-kube-api-access-sfmm9\") pod \"maas-api-96979fff8-m7j7d\" (UID: \"2311a276-c41e-4079-b677-bc3e45864ebc\") " pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.129977 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.129892 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:26.286257 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.286219 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-96979fff8-m7j7d"] Jul 22 04:38:26.289640 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:38:26.289597 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2311a276_c41e_4079_b677_bc3e45864ebc.slice/crio-5d967719b098667ebf33a007c8343fe3c4e67e7bf0b3e42cb0af184ed6a8ae42 WatchSource:0}: Error finding container 5d967719b098667ebf33a007c8343fe3c4e67e7bf0b3e42cb0af184ed6a8ae42: Status 404 returned error can't find the container with id 5d967719b098667ebf33a007c8343fe3c4e67e7bf0b3e42cb0af184ed6a8ae42 Jul 22 04:38:26.436956 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.436926 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l9gfw"] Jul 22 04:38:26.443644 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.443616 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.448642 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.448488 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 22 04:38:26.448642 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.448488 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-l75n7\"" Jul 22 04:38:26.451603 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.451573 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l9gfw"] Jul 22 04:38:26.539231 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.539196 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" event={"ID":"2311a276-c41e-4079-b677-bc3e45864ebc","Type":"ContainerStarted","Data":"5d967719b098667ebf33a007c8343fe3c4e67e7bf0b3e42cb0af184ed6a8ae42"} Jul 22 04:38:26.617632 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.617598 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18d6e2c-20cd-4219-be78-23381e89f5d5-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l9gfw\" (UID: \"f18d6e2c-20cd-4219-be78-23381e89f5d5\") " pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.617810 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.617687 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkmw8\" (UniqueName: \"kubernetes.io/projected/f18d6e2c-20cd-4219-be78-23381e89f5d5-kube-api-access-wkmw8\") pod \"payload-processing-5bf47cd988-l9gfw\" (UID: \"f18d6e2c-20cd-4219-be78-23381e89f5d5\") " pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.719202 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.719109 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkmw8\" (UniqueName: \"kubernetes.io/projected/f18d6e2c-20cd-4219-be78-23381e89f5d5-kube-api-access-wkmw8\") pod \"payload-processing-5bf47cd988-l9gfw\" (UID: \"f18d6e2c-20cd-4219-be78-23381e89f5d5\") " pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.719202 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.719168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18d6e2c-20cd-4219-be78-23381e89f5d5-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l9gfw\" (UID: \"f18d6e2c-20cd-4219-be78-23381e89f5d5\") " pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.719845 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.719822 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f18d6e2c-20cd-4219-be78-23381e89f5d5-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l9gfw\" (UID: \"f18d6e2c-20cd-4219-be78-23381e89f5d5\") " pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.726868 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.726838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkmw8\" (UniqueName: \"kubernetes.io/projected/f18d6e2c-20cd-4219-be78-23381e89f5d5-kube-api-access-wkmw8\") pod \"payload-processing-5bf47cd988-l9gfw\" (UID: \"f18d6e2c-20cd-4219-be78-23381e89f5d5\") " pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.755887 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.755853 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:26.913137 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:26.913098 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l9gfw"] Jul 22 04:38:26.916228 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:38:26.916197 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf18d6e2c_20cd_4219_be78_23381e89f5d5.slice/crio-9793ac789e3fd7ed86ff24068113d9d40c5be77bda7bdbc6bf25f8d746011e5c WatchSource:0}: Error finding container 9793ac789e3fd7ed86ff24068113d9d40c5be77bda7bdbc6bf25f8d746011e5c: Status 404 returned error can't find the container with id 9793ac789e3fd7ed86ff24068113d9d40c5be77bda7bdbc6bf25f8d746011e5c Jul 22 04:38:27.549663 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:27.549626 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" event={"ID":"f18d6e2c-20cd-4219-be78-23381e89f5d5","Type":"ContainerStarted","Data":"9793ac789e3fd7ed86ff24068113d9d40c5be77bda7bdbc6bf25f8d746011e5c"} Jul 22 04:38:28.558230 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:28.558191 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" event={"ID":"2311a276-c41e-4079-b677-bc3e45864ebc","Type":"ContainerStarted","Data":"df45f9832d42e28f4b7e2cb5322013bad8c06398915f7a33b2f47d608e0dc8cc"} Jul 22 04:38:28.558712 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:28.558280 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:28.575485 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:28.575426 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" podStartSLOduration=2.24951259 podStartE2EDuration="3.575410122s" totalImagesPullingTime="1.325897532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:28.572265997 +0000 UTC m=+832.661545171" watchObservedRunningTime="2026-07-22 04:38:28.575410122 +0000 UTC m=+832.664689296" Jul 22 04:38:29.565487 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:29.565449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" event={"ID":"f18d6e2c-20cd-4219-be78-23381e89f5d5","Type":"ContainerStarted","Data":"c2762869daa265cd24bd9cc86d7253a1fdf6d8effcea9fbef262b26952b5f600"} Jul 22 04:38:29.565919 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:29.565768 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:29.580944 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:29.580894 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" podStartSLOduration=1.858077635 podStartE2EDuration="3.580879195s" totalImagesPullingTime="1.72280156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:29.579055049 +0000 UTC m=+833.668334222" watchObservedRunningTime="2026-07-22 04:38:29.580879195 +0000 UTC m=+833.670158366" Jul 22 04:38:34.571862 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:34.571824 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-96979fff8-m7j7d" Jul 22 04:38:40.574138 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.574105 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-l9gfw" Jul 22 04:38:40.860880 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.860803 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm"] Jul 22 04:38:40.866460 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.866433 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:40.869803 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.869776 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 04:38:40.869980 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.869776 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 04:38:40.870131 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.869852 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qkdmg\"" Jul 22 04:38:40.870257 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.869905 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 04:38:40.871295 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.871269 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm"] Jul 22 04:38:40.950006 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.949970 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26dvk\" (UniqueName: \"kubernetes.io/projected/33e9e8d9-f408-4a63-adee-8105161e533d-kube-api-access-26dvk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:40.950006 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.950007 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:40.950250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.950024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:40.950250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.950045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:40.950250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.950141 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:40.950250 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:40.950188 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e9e8d9-f408-4a63-adee-8105161e533d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051459 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e9e8d9-f408-4a63-adee-8105161e533d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051652 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051487 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26dvk\" (UniqueName: \"kubernetes.io/projected/33e9e8d9-f408-4a63-adee-8105161e533d-kube-api-access-26dvk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051652 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051520 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051652 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051544 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051807 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051807 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.051962 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.051939 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.052115 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.052090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.052187 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.052121 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.053764 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.053745 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33e9e8d9-f408-4a63-adee-8105161e533d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.054061 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.054039 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33e9e8d9-f408-4a63-adee-8105161e533d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.058464 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.058446 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26dvk\" (UniqueName: \"kubernetes.io/projected/33e9e8d9-f408-4a63-adee-8105161e533d-kube-api-access-26dvk\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm\" (UID: \"33e9e8d9-f408-4a63-adee-8105161e533d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.179482 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.179445 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:41.314506 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.314479 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm"] Jul 22 04:38:41.316731 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:38:41.316702 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33e9e8d9_f408_4a63_adee_8105161e533d.slice/crio-d90f680d097ed5b24f9240bd9189cfaa43ee3d1cfbc2715a295e4e8e124d412f WatchSource:0}: Error finding container d90f680d097ed5b24f9240bd9189cfaa43ee3d1cfbc2715a295e4e8e124d412f: Status 404 returned error can't find the container with id d90f680d097ed5b24f9240bd9189cfaa43ee3d1cfbc2715a295e4e8e124d412f Jul 22 04:38:41.639992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:41.639767 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" event={"ID":"33e9e8d9-f408-4a63-adee-8105161e533d","Type":"ContainerStarted","Data":"d90f680d097ed5b24f9240bd9189cfaa43ee3d1cfbc2715a295e4e8e124d412f"} Jul 22 04:38:42.647351 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:42.647311 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" event={"ID":"33e9e8d9-f408-4a63-adee-8105161e533d","Type":"ContainerStarted","Data":"f6f34c80771954668b46c80d9abe2c79efcdec39c469fecfa615dc1e842d241e"} Jul 22 04:38:42.686952 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:42.686888 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" podStartSLOduration=1.46641654 podStartE2EDuration="2.686871301s" totalImagesPullingTime="1.220454761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:42.685286682 +0000 UTC m=+846.774565879" watchObservedRunningTime="2026-07-22 04:38:42.686871301 +0000 UTC m=+846.776150473" Jul 22 04:38:51.180117 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:51.180054 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:51.180669 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:51.180147 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:51.192199 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:51.192173 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:51.706737 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:51.706700 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm" Jul 22 04:38:56.170609 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.170572 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5"] Jul 22 04:38:56.175027 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.175003 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.177514 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.177483 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 04:38:56.184392 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.184367 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5"] Jul 22 04:38:56.288455 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.288410 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.288455 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.288461 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.288664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.288479 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.288664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.288541 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.288664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.288592 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.288664 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.288614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xz5qk\" (UniqueName: \"kubernetes.io/projected/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-kube-api-access-xz5qk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389010 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.388980 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389010 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389022 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xz5qk\" (UniqueName: \"kubernetes.io/projected/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-kube-api-access-xz5qk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389294 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389118 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389294 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389294 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389182 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389464 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389464 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389452 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389572 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389516 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.389572 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.389540 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.391482 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.391461 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.391588 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.391558 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.403150 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.403130 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xz5qk\" (UniqueName: \"kubernetes.io/projected/3922e9e6-49c5-4b5c-a25e-fd673e714e0e-kube-api-access-xz5qk\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5\" (UID: \"3922e9e6-49c5-4b5c-a25e-fd673e714e0e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.485879 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.485798 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:38:56.615859 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.615834 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5"] Jul 22 04:38:56.617318 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:38:56.617291 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3922e9e6_49c5_4b5c_a25e_fd673e714e0e.slice/crio-4d182d31d711df138eedf1635c9fe8cc22cc1ced9be3dfdf52ceb76a1c54054d WatchSource:0}: Error finding container 4d182d31d711df138eedf1635c9fe8cc22cc1ced9be3dfdf52ceb76a1c54054d: Status 404 returned error can't find the container with id 4d182d31d711df138eedf1635c9fe8cc22cc1ced9be3dfdf52ceb76a1c54054d Jul 22 04:38:56.723726 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:56.723692 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" event={"ID":"3922e9e6-49c5-4b5c-a25e-fd673e714e0e","Type":"ContainerStarted","Data":"4d182d31d711df138eedf1635c9fe8cc22cc1ced9be3dfdf52ceb76a1c54054d"} Jul 22 04:38:57.731359 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:57.731324 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" event={"ID":"3922e9e6-49c5-4b5c-a25e-fd673e714e0e","Type":"ContainerStarted","Data":"c54bd791a9ce70102f21ba0ac61d235ac0593a2bd628706fee2aff128337c064"} Jul 22 04:38:57.751788 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:38:57.751737 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" podStartSLOduration=1.582081347 podStartE2EDuration="1.751721028s" totalImagesPullingTime="169.639681ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:38:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:38:57.749329802 +0000 UTC m=+861.838609001" watchObservedRunningTime="2026-07-22 04:38:57.751721028 +0000 UTC m=+861.841000199" Jul 22 04:39:06.486915 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:06.486875 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:39:06.487366 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:06.486948 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:39:06.499238 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:06.499213 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:39:06.801935 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:06.801849 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5" Jul 22 04:39:19.268899 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.268861 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5"] Jul 22 04:39:19.274247 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.274229 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.276672 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.276651 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 22 04:39:19.282017 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.281801 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5"] Jul 22 04:39:19.401714 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.401679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.401714 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.401717 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv7s4\" (UniqueName: \"kubernetes.io/projected/36ef2bf8-c50e-4302-9e93-f130e649522f-kube-api-access-hv7s4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.401996 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.401744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.401996 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.401860 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.401996 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.401900 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ef2bf8-c50e-4302-9e93-f130e649522f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.402173 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.402004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503426 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503391 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503431 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv7s4\" (UniqueName: \"kubernetes.io/projected/36ef2bf8-c50e-4302-9e93-f130e649522f-kube-api-access-hv7s4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503462 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503506 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ef2bf8-c50e-4302-9e93-f130e649522f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503614 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503608 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503778 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503869 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.503907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.503884 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.505867 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.505840 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ef2bf8-c50e-4302-9e93-f130e649522f-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.506106 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.506088 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ef2bf8-c50e-4302-9e93-f130e649522f-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.510940 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.510922 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv7s4\" (UniqueName: \"kubernetes.io/projected/36ef2bf8-c50e-4302-9e93-f130e649522f-kube-api-access-hv7s4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-gksb5\" (UID: \"36ef2bf8-c50e-4302-9e93-f130e649522f\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.584871 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.584796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:19.714248 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.714223 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5"] Jul 22 04:39:19.715862 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:39:19.715822 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36ef2bf8_c50e_4302_9e93_f130e649522f.slice/crio-581a9346a62a15ca14f48400d7b90757ab4c229cebdf10a8d4be086c89b79b22 WatchSource:0}: Error finding container 581a9346a62a15ca14f48400d7b90757ab4c229cebdf10a8d4be086c89b79b22: Status 404 returned error can't find the container with id 581a9346a62a15ca14f48400d7b90757ab4c229cebdf10a8d4be086c89b79b22 Jul 22 04:39:19.860467 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:19.860369 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" event={"ID":"36ef2bf8-c50e-4302-9e93-f130e649522f","Type":"ContainerStarted","Data":"581a9346a62a15ca14f48400d7b90757ab4c229cebdf10a8d4be086c89b79b22"} Jul 22 04:39:20.866976 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:20.866875 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" event={"ID":"36ef2bf8-c50e-4302-9e93-f130e649522f","Type":"ContainerStarted","Data":"d4527dc7463acac3460dbc0e816ce2363beda2ab157cb9d9adb24996b17a016e"} Jul 22 04:39:20.885210 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:20.885163 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" podStartSLOduration=1.711861947 podStartE2EDuration="1.885148825s" totalImagesPullingTime="173.286878ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:39:20.881929675 +0000 UTC m=+884.971208863" watchObservedRunningTime="2026-07-22 04:39:20.885148825 +0000 UTC m=+884.974427994" Jul 22 04:39:29.585244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:29.585163 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:29.585244 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:29.585210 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:29.597408 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:29.597380 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:39:29.926902 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:39:29.926870 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-gksb5" Jul 22 04:40:07.133411 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.133371 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-665b5547bd-7krsg"] Jul 22 04:40:07.136846 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.136828 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.145869 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.145843 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-665b5547bd-7krsg"] Jul 22 04:40:07.231784 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.231742 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czpfs\" (UniqueName: \"kubernetes.io/projected/26e70862-18e5-4a6a-9cd3-89da4a8711f9-kube-api-access-czpfs\") pod \"authorino-665b5547bd-7krsg\" (UID: \"26e70862-18e5-4a6a-9cd3-89da4a8711f9\") " pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.231956 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.231794 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/26e70862-18e5-4a6a-9cd3-89da4a8711f9-tls-cert\") pod \"authorino-665b5547bd-7krsg\" (UID: \"26e70862-18e5-4a6a-9cd3-89da4a8711f9\") " pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.333289 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.333256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czpfs\" (UniqueName: \"kubernetes.io/projected/26e70862-18e5-4a6a-9cd3-89da4a8711f9-kube-api-access-czpfs\") pod \"authorino-665b5547bd-7krsg\" (UID: \"26e70862-18e5-4a6a-9cd3-89da4a8711f9\") " pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.333447 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.333306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/26e70862-18e5-4a6a-9cd3-89da4a8711f9-tls-cert\") pod \"authorino-665b5547bd-7krsg\" (UID: \"26e70862-18e5-4a6a-9cd3-89da4a8711f9\") " pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.335852 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.335826 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/26e70862-18e5-4a6a-9cd3-89da4a8711f9-tls-cert\") pod \"authorino-665b5547bd-7krsg\" (UID: \"26e70862-18e5-4a6a-9cd3-89da4a8711f9\") " pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.346806 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.346777 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-czpfs\" (UniqueName: \"kubernetes.io/projected/26e70862-18e5-4a6a-9cd3-89da4a8711f9-kube-api-access-czpfs\") pod \"authorino-665b5547bd-7krsg\" (UID: \"26e70862-18e5-4a6a-9cd3-89da4a8711f9\") " pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.446298 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.446262 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-665b5547bd-7krsg" Jul 22 04:40:07.578558 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:07.578530 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-665b5547bd-7krsg"] Jul 22 04:40:07.580102 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:40:07.580046 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod26e70862_18e5_4a6a_9cd3_89da4a8711f9.slice/crio-f787be0a235b4fe776ba34856037e70e89fda76c452b714417ba453e0fbee7c3 WatchSource:0}: Error finding container f787be0a235b4fe776ba34856037e70e89fda76c452b714417ba453e0fbee7c3: Status 404 returned error can't find the container with id f787be0a235b4fe776ba34856037e70e89fda76c452b714417ba453e0fbee7c3 Jul 22 04:40:08.122116 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:08.122081 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-665b5547bd-7krsg" event={"ID":"26e70862-18e5-4a6a-9cd3-89da4a8711f9","Type":"ContainerStarted","Data":"f787be0a235b4fe776ba34856037e70e89fda76c452b714417ba453e0fbee7c3"} Jul 22 04:40:09.131327 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.131283 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-665b5547bd-7krsg" event={"ID":"26e70862-18e5-4a6a-9cd3-89da4a8711f9","Type":"ContainerStarted","Data":"98e4592a5642bc7335019fd7565f3259dde1d96ce78941e8b976a6e6e49761f8"} Jul 22 04:40:09.150694 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.150636 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-665b5547bd-7krsg" podStartSLOduration=1.221882212 podStartE2EDuration="2.150614047s" totalImagesPullingTime="928.731835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:40:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:40:09.147737869 +0000 UTC m=+933.237017040" watchObservedRunningTime="2026-07-22 04:40:09.150614047 +0000 UTC m=+933.239893221" Jul 22 04:40:09.187561 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.187521 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659977b69c-94chr"] Jul 22 04:40:09.187782 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.187749 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-659977b69c-94chr" podUID="28e02621-336a-4b7b-a6a0-633750c922c7" containerName="authorino" containerID="cri-o://8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7" gracePeriod=30 Jul 22 04:40:09.470669 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.470637 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:40:09.655269 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.655234 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28e02621-336a-4b7b-a6a0-633750c922c7-tls-cert\") pod \"28e02621-336a-4b7b-a6a0-633750c922c7\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " Jul 22 04:40:09.655446 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.655297 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d5kw\" (UniqueName: \"kubernetes.io/projected/28e02621-336a-4b7b-a6a0-633750c922c7-kube-api-access-2d5kw\") pod \"28e02621-336a-4b7b-a6a0-633750c922c7\" (UID: \"28e02621-336a-4b7b-a6a0-633750c922c7\") " Jul 22 04:40:09.657437 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.657399 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28e02621-336a-4b7b-a6a0-633750c922c7-kube-api-access-2d5kw" (OuterVolumeSpecName: "kube-api-access-2d5kw") pod "28e02621-336a-4b7b-a6a0-633750c922c7" (UID: "28e02621-336a-4b7b-a6a0-633750c922c7"). InnerVolumeSpecName "kube-api-access-2d5kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:40:09.666132 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.666095 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28e02621-336a-4b7b-a6a0-633750c922c7-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "28e02621-336a-4b7b-a6a0-633750c922c7" (UID: "28e02621-336a-4b7b-a6a0-633750c922c7"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:40:09.756689 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.756645 2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28e02621-336a-4b7b-a6a0-633750c922c7-tls-cert\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:40:09.756689 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:09.756676 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d5kw\" (UniqueName: \"kubernetes.io/projected/28e02621-336a-4b7b-a6a0-633750c922c7-kube-api-access-2d5kw\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:40:10.138262 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.138163 2566 generic.go:356] "Generic (PLEG): container finished" podID="28e02621-336a-4b7b-a6a0-633750c922c7" containerID="8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7" exitCode=0 Jul 22 04:40:10.138262 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.138199 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-659977b69c-94chr" Jul 22 04:40:10.138262 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.138251 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659977b69c-94chr" event={"ID":"28e02621-336a-4b7b-a6a0-633750c922c7","Type":"ContainerDied","Data":"8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7"} Jul 22 04:40:10.138907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.138297 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-659977b69c-94chr" event={"ID":"28e02621-336a-4b7b-a6a0-633750c922c7","Type":"ContainerDied","Data":"31ec16df72c472e978164c87b2d3de41c1682e609f35f1465b69b4cd8b3ed8fb"} Jul 22 04:40:10.138907 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.138319 2566 scope.go:117] "RemoveContainer" containerID="8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7" Jul 22 04:40:10.148986 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.148964 2566 scope.go:117] "RemoveContainer" containerID="8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7" Jul 22 04:40:10.149315 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:40:10.149295 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7\": container with ID starting with 8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7 not found: ID does not exist" containerID="8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7" Jul 22 04:40:10.149388 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.149324 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7"} err="failed to get container status \"8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7\": rpc error: code = NotFound desc = could not find container \"8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7\": container with ID starting with 8a318d5fd3b00b6019d3717b145d8a5c40fbc0b409e6ab12daca0d1bf935eaf7 not found: ID does not exist" Jul 22 04:40:10.161227 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.161200 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-659977b69c-94chr"] Jul 22 04:40:10.165005 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.164980 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-659977b69c-94chr"] Jul 22 04:40:10.931976 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:40:10.931941 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28e02621-336a-4b7b-a6a0-633750c922c7" path="/var/lib/kubelet/pods/28e02621-336a-4b7b-a6a0-633750c922c7/volumes" Jul 22 04:45:00.143531 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.143492 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5"] Jul 22 04:45:00.145514 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.143889 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28e02621-336a-4b7b-a6a0-633750c922c7" containerName="authorino" Jul 22 04:45:00.145514 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.143900 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="28e02621-336a-4b7b-a6a0-633750c922c7" containerName="authorino" Jul 22 04:45:00.145514 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.143962 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="28e02621-336a-4b7b-a6a0-633750c922c7" containerName="authorino" Jul 22 04:45:00.146993 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.146975 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" Jul 22 04:45:00.149299 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.149284 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-65v9v\"" Jul 22 04:45:00.156483 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.156466 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" Jul 22 04:45:00.159915 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.159891 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5"] Jul 22 04:45:00.294464 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.294430 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5"] Jul 22 04:45:00.295341 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:45:00.295309 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42272ed7_224e_4600_b117_14a88c8912cc.slice/crio-9f52506e65cd303d8f9ffdcd66be6f655c63f3cbf74fff12fa3291443000999e WatchSource:0}: Error finding container 9f52506e65cd303d8f9ffdcd66be6f655c63f3cbf74fff12fa3291443000999e: Status 404 returned error can't find the container with id 9f52506e65cd303d8f9ffdcd66be6f655c63f3cbf74fff12fa3291443000999e Jul 22 04:45:00.297562 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.297546 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:45:00.744199 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:00.744161 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" event={"ID":"42272ed7-224e-4600-b117-14a88c8912cc","Type":"ContainerStarted","Data":"9f52506e65cd303d8f9ffdcd66be6f655c63f3cbf74fff12fa3291443000999e"} Jul 22 04:45:02.758362 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:02.758320 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" event={"ID":"42272ed7-224e-4600-b117-14a88c8912cc","Type":"ContainerStarted","Data":"db1bc7c7b13918d857b6f32c0797e3380ba73cb403466fca30683e0fba46d6ad"} Jul 22 04:45:02.774683 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:02.774622 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" podStartSLOduration=0.531664076 podStartE2EDuration="2.774607085s" totalImagesPullingTime="2.242943009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:45:02.771439924 +0000 UTC m=+1226.860719096" watchObservedRunningTime="2026-07-22 04:45:02.774607085 +0000 UTC m=+1226.863886257" Jul 22 04:45:03.764978 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:03.764941 2566 generic.go:356] "Generic (PLEG): container finished" podID="42272ed7-224e-4600-b117-14a88c8912cc" containerID="db1bc7c7b13918d857b6f32c0797e3380ba73cb403466fca30683e0fba46d6ad" exitCode=0 Jul 22 04:45:03.765404 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:03.765028 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" event={"ID":"42272ed7-224e-4600-b117-14a88c8912cc","Type":"ContainerDied","Data":"db1bc7c7b13918d857b6f32c0797e3380ba73cb403466fca30683e0fba46d6ad"} Jul 22 04:45:04.899739 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:04.899715 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" Jul 22 04:45:05.777768 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:05.777740 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" Jul 22 04:45:05.777768 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:05.777759 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744925-94dh5" event={"ID":"42272ed7-224e-4600-b117-14a88c8912cc","Type":"ContainerDied","Data":"9f52506e65cd303d8f9ffdcd66be6f655c63f3cbf74fff12fa3291443000999e"} Jul 22 04:45:05.777990 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:45:05.777805 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f52506e65cd303d8f9ffdcd66be6f655c63f3cbf74fff12fa3291443000999e" Jul 22 04:55:04.420662 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.420616 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2"] Jul 22 04:55:04.421279 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.421258 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42272ed7-224e-4600-b117-14a88c8912cc" containerName="cleanup" Jul 22 04:55:04.421328 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.421285 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="42272ed7-224e-4600-b117-14a88c8912cc" containerName="cleanup" Jul 22 04:55:04.421433 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.421416 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="42272ed7-224e-4600-b117-14a88c8912cc" containerName="cleanup" Jul 22 04:55:04.424778 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.424755 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.431593 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.431560 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e1453556-serving-cert\"" Jul 22 04:55:04.434181 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.434140 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2"] Jul 22 04:55:04.463757 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.463719 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8m4\" (UniqueName: \"kubernetes.io/projected/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-kube-api-access-9l8m4\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.463974 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.463775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-openshift-service-ca\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.463974 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.463913 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.565034 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.564986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8m4\" (UniqueName: \"kubernetes.io/projected/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-kube-api-access-9l8m4\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.565273 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.565064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-openshift-service-ca\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.565273 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.565167 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.565415 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:55:04.565289 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-serving-cert: secret "maas-api-e2e-ait-e1453556-serving-cert" not found Jul 22 04:55:04.565415 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:55:04.565355 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls podName:0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e nodeName:}" failed. No retries permitted until 2026-07-22 04:55:05.065334792 +0000 UTC m=+1829.154613947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls") pod "maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" (UID: "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e") : secret "maas-api-e2e-ait-e1453556-serving-cert" not found Jul 22 04:55:04.565860 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.565833 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-openshift-service-ca\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:04.573720 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:04.573694 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8m4\" (UniqueName: \"kubernetes.io/projected/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-kube-api-access-9l8m4\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:05.070363 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:05.070326 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:05.072941 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:05.072914 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls\") pod \"maas-api-e2e-ait-e1453556-788db9cbc-qh2b2\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:05.336924 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:05.336822 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:05.495001 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:05.494969 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2"] Jul 22 04:55:05.496530 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:55:05.496500 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a74561c_f7a2_40fa_a3cd_2c9d1e930f3e.slice/crio-3e78d8fb67cb01d347590ee5a83c40130893d9771b01d5158b7620bd740cd20f WatchSource:0}: Error finding container 3e78d8fb67cb01d347590ee5a83c40130893d9771b01d5158b7620bd740cd20f: Status 404 returned error can't find the container with id 3e78d8fb67cb01d347590ee5a83c40130893d9771b01d5158b7620bd740cd20f Jul 22 04:55:05.497862 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:05.497843 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 04:55:06.115749 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:06.115714 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" event={"ID":"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e","Type":"ContainerStarted","Data":"1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e"} Jul 22 04:55:06.115934 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:06.115768 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" event={"ID":"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e","Type":"ContainerStarted","Data":"3e78d8fb67cb01d347590ee5a83c40130893d9771b01d5158b7620bd740cd20f"} Jul 22 04:55:06.115934 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:06.115915 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:06.132171 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:06.132062 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" podStartSLOduration=1.690009165 podStartE2EDuration="2.132042875s" totalImagesPullingTime="442.03371ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:55:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:55:06.131793396 +0000 UTC m=+1830.221072592" watchObservedRunningTime="2026-07-22 04:55:06.132042875 +0000 UTC m=+1830.221322049" Jul 22 04:55:12.127522 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:12.127492 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:40.257734 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.257695 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2"] Jul 22 04:55:40.258347 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.257965 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" podUID="0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" containerName="maas-api" containerID="cri-o://1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e" gracePeriod=30 Jul 22 04:55:40.503637 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.503611 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:40.695136 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.695063 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-openshift-service-ca\") pod \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " Jul 22 04:55:40.695313 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.695166 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l8m4\" (UniqueName: \"kubernetes.io/projected/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-kube-api-access-9l8m4\") pod \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " Jul 22 04:55:40.695313 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.695208 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls\") pod \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\" (UID: \"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e\") " Jul 22 04:55:40.695444 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.695418 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" (UID: "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:55:40.697410 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.697386 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" (UID: "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:55:40.697554 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.697530 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-kube-api-access-9l8m4" (OuterVolumeSpecName: "kube-api-access-9l8m4") pod "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" (UID: "0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e"). InnerVolumeSpecName "kube-api-access-9l8m4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:55:40.796495 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.796458 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:55:40.796495 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.796491 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:55:40.796495 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:40.796501 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l8m4\" (UniqueName: \"kubernetes.io/projected/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e-kube-api-access-9l8m4\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:55:41.312527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.312491 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" containerID="1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e" exitCode=0 Jul 22 04:55:41.312945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.312540 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" Jul 22 04:55:41.312945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.312581 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" event={"ID":"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e","Type":"ContainerDied","Data":"1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e"} Jul 22 04:55:41.312945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.312638 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2" event={"ID":"0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e","Type":"ContainerDied","Data":"3e78d8fb67cb01d347590ee5a83c40130893d9771b01d5158b7620bd740cd20f"} Jul 22 04:55:41.312945 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.312667 2566 scope.go:117] "RemoveContainer" containerID="1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e" Jul 22 04:55:41.321826 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.321801 2566 scope.go:117] "RemoveContainer" containerID="1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e" Jul 22 04:55:41.322100 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:55:41.322076 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e\": container with ID starting with 1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e not found: ID does not exist" containerID="1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e" Jul 22 04:55:41.322171 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.322107 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e"} err="failed to get container status \"1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e\": rpc error: code = NotFound desc = could not find container \"1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e\": container with ID starting with 1dfc94e0b6c2930922c22827e12381dda0b9f2da82cc6af16e1ec6529c3a6a0e not found: ID does not exist" Jul 22 04:55:41.329520 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.329495 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2"] Jul 22 04:55:41.333172 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:41.333152 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e1453556-788db9cbc-qh2b2"] Jul 22 04:55:42.932287 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:42.932249 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" path="/var/lib/kubelet/pods/0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e/volumes" Jul 22 04:55:48.158392 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.158346 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc"] Jul 22 04:55:48.170686 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.170649 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" containerName="maas-api" Jul 22 04:55:48.170964 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.170941 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" containerName="maas-api" Jul 22 04:55:48.172429 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.171990 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a74561c-f7a2-40fa-a3cd-2c9d1e930f3e" containerName="maas-api" Jul 22 04:55:48.178147 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.178123 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc"] Jul 22 04:55:48.178264 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.178232 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.180640 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.180618 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-90fa15ea-serving-cert\"" Jul 22 04:55:48.253644 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.253608 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.253840 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.253662 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-openshift-service-ca\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.253840 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.253737 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hgwp\" (UniqueName: \"kubernetes.io/projected/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-kube-api-access-2hgwp\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.355306 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.355268 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.355510 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.355319 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-openshift-service-ca\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.355510 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.355361 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hgwp\" (UniqueName: \"kubernetes.io/projected/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-kube-api-access-2hgwp\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.355510 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:55:48.355442 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-serving-cert: secret "maas-api-e2e-migrate-90fa15ea-serving-cert" not found Jul 22 04:55:48.355685 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:55:48.355531 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls podName:b8ea28f4-da0f-437d-8a37-7211e3ab39a8 nodeName:}" failed. No retries permitted until 2026-07-22 04:55:48.855508934 +0000 UTC m=+1872.944788089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls") pod "maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" (UID: "b8ea28f4-da0f-437d-8a37-7211e3ab39a8") : secret "maas-api-e2e-migrate-90fa15ea-serving-cert" not found Jul 22 04:55:48.356110 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.356063 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-openshift-service-ca\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.363497 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.363470 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hgwp\" (UniqueName: \"kubernetes.io/projected/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-kube-api-access-2hgwp\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.860569 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.860536 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:48.863038 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:48.863017 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls\") pod \"maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:49.090995 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:49.090945 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:49.228353 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:49.228325 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc"] Jul 22 04:55:49.230182 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:55:49.230152 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ea28f4_da0f_437d_8a37_7211e3ab39a8.slice/crio-ce66503659a01e9041050d96ae40211e13b7c8785de0fd11ed5a5862e814440f WatchSource:0}: Error finding container ce66503659a01e9041050d96ae40211e13b7c8785de0fd11ed5a5862e814440f: Status 404 returned error can't find the container with id ce66503659a01e9041050d96ae40211e13b7c8785de0fd11ed5a5862e814440f Jul 22 04:55:49.359093 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:49.359032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" event={"ID":"b8ea28f4-da0f-437d-8a37-7211e3ab39a8","Type":"ContainerStarted","Data":"ce66503659a01e9041050d96ae40211e13b7c8785de0fd11ed5a5862e814440f"} Jul 22 04:55:50.366293 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:50.366254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" event={"ID":"b8ea28f4-da0f-437d-8a37-7211e3ab39a8","Type":"ContainerStarted","Data":"3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06"} Jul 22 04:55:50.368630 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:50.366321 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:55:50.383888 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:50.383836 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" podStartSLOduration=1.974266617 podStartE2EDuration="2.3838195s" totalImagesPullingTime="409.552883ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:55:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:55:50.38174846 +0000 UTC m=+1874.471027664" watchObservedRunningTime="2026-07-22 04:55:50.3838195 +0000 UTC m=+1874.473098672" Jul 22 04:55:56.378325 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:55:56.378292 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:56:23.601432 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.601393 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc"] Jul 22 04:56:23.601925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.601651 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" podUID="b8ea28f4-da0f-437d-8a37-7211e3ab39a8" containerName="maas-api" containerID="cri-o://3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06" gracePeriod=30 Jul 22 04:56:23.849788 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.849761 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:56:23.975241 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.975201 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-openshift-service-ca\") pod \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " Jul 22 04:56:23.975241 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.975245 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls\") pod \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " Jul 22 04:56:23.975497 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.975328 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hgwp\" (UniqueName: \"kubernetes.io/projected/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-kube-api-access-2hgwp\") pod \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\" (UID: \"b8ea28f4-da0f-437d-8a37-7211e3ab39a8\") " Jul 22 04:56:23.975636 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.975611 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b8ea28f4-da0f-437d-8a37-7211e3ab39a8" (UID: "b8ea28f4-da0f-437d-8a37-7211e3ab39a8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:56:23.979106 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.977711 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-kube-api-access-2hgwp" (OuterVolumeSpecName: "kube-api-access-2hgwp") pod "b8ea28f4-da0f-437d-8a37-7211e3ab39a8" (UID: "b8ea28f4-da0f-437d-8a37-7211e3ab39a8"). InnerVolumeSpecName "kube-api-access-2hgwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:56:23.979344 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:23.979280 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b8ea28f4-da0f-437d-8a37-7211e3ab39a8" (UID: "b8ea28f4-da0f-437d-8a37-7211e3ab39a8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:56:24.076965 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.076925 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:56:24.076965 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.076956 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:56:24.076965 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.076966 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hgwp\" (UniqueName: \"kubernetes.io/projected/b8ea28f4-da0f-437d-8a37-7211e3ab39a8-kube-api-access-2hgwp\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:56:24.558122 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.558084 2566 generic.go:356] "Generic (PLEG): container finished" podID="b8ea28f4-da0f-437d-8a37-7211e3ab39a8" containerID="3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06" exitCode=0 Jul 22 04:56:24.558354 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.558151 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" Jul 22 04:56:24.558354 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.558167 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" event={"ID":"b8ea28f4-da0f-437d-8a37-7211e3ab39a8","Type":"ContainerDied","Data":"3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06"} Jul 22 04:56:24.558354 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.558256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc" event={"ID":"b8ea28f4-da0f-437d-8a37-7211e3ab39a8","Type":"ContainerDied","Data":"ce66503659a01e9041050d96ae40211e13b7c8785de0fd11ed5a5862e814440f"} Jul 22 04:56:24.558354 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.558292 2566 scope.go:117] "RemoveContainer" containerID="3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06" Jul 22 04:56:24.568375 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.568357 2566 scope.go:117] "RemoveContainer" containerID="3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06" Jul 22 04:56:24.568645 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:56:24.568626 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06\": container with ID starting with 3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06 not found: ID does not exist" containerID="3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06" Jul 22 04:56:24.568705 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.568660 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06"} err="failed to get container status \"3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06\": rpc error: code = NotFound desc = could not find container \"3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06\": container with ID starting with 3f1367964384f191f39ea10fd84e56206eaa180b881cdfeb2cb0cf041dc56f06 not found: ID does not exist" Jul 22 04:56:24.579972 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.579950 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc"] Jul 22 04:56:24.583119 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.583088 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-90fa15ea-5587cfd4f6-fvcjc"] Jul 22 04:56:24.931992 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:24.931959 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8ea28f4-da0f-437d-8a37-7211e3ab39a8" path="/var/lib/kubelet/pods/b8ea28f4-da0f-437d-8a37-7211e3ab39a8/volumes" Jul 22 04:56:31.857013 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.856982 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp"] Jul 22 04:56:31.857425 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.857411 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8ea28f4-da0f-437d-8a37-7211e3ab39a8" containerName="maas-api" Jul 22 04:56:31.857474 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.857427 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8ea28f4-da0f-437d-8a37-7211e3ab39a8" containerName="maas-api" Jul 22 04:56:31.857543 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.857533 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8ea28f4-da0f-437d-8a37-7211e3ab39a8" containerName="maas-api" Jul 22 04:56:31.862129 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.862104 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:31.864306 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.864282 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e7529972-serving-cert\"" Jul 22 04:56:31.869930 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.869905 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp"] Jul 22 04:56:31.954371 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.954343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vsp2\" (UniqueName: \"kubernetes.io/projected/d392003a-2edc-4fe7-a8f1-469095b0d3ea-kube-api-access-4vsp2\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:31.954536 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.954397 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:31.954536 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:31.954487 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d392003a-2edc-4fe7-a8f1-469095b0d3ea-openshift-service-ca\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.055376 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:32.055340 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vsp2\" (UniqueName: \"kubernetes.io/projected/d392003a-2edc-4fe7-a8f1-469095b0d3ea-kube-api-access-4vsp2\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.055540 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:32.055391 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.055540 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:32.055430 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d392003a-2edc-4fe7-a8f1-469095b0d3ea-openshift-service-ca\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.055667 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:56:32.055541 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-serving-cert: secret "maas-api-e2e-ait-e7529972-serving-cert" not found Jul 22 04:56:32.055667 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:56:32.055616 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls podName:d392003a-2edc-4fe7-a8f1-469095b0d3ea nodeName:}" failed. No retries permitted until 2026-07-22 04:56:32.555596155 +0000 UTC m=+1916.644875324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls") pod "maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" (UID: "d392003a-2edc-4fe7-a8f1-469095b0d3ea") : secret "maas-api-e2e-ait-e7529972-serving-cert" not found Jul 22 04:56:32.055996 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:32.055979 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d392003a-2edc-4fe7-a8f1-469095b0d3ea-openshift-service-ca\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.083687 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:32.083651 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vsp2\" (UniqueName: \"kubernetes.io/projected/d392003a-2edc-4fe7-a8f1-469095b0d3ea-kube-api-access-4vsp2\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.559848 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:32.559812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:32.560032 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:56:32.559956 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-serving-cert: secret "maas-api-e2e-ait-e7529972-serving-cert" not found Jul 22 04:56:32.560032 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:56:32.560021 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls podName:d392003a-2edc-4fe7-a8f1-469095b0d3ea nodeName:}" failed. No retries permitted until 2026-07-22 04:56:33.560006868 +0000 UTC m=+1917.649286021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls") pod "maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" (UID: "d392003a-2edc-4fe7-a8f1-469095b0d3ea") : secret "maas-api-e2e-ait-e7529972-serving-cert" not found Jul 22 04:56:33.570932 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:33.570891 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:33.573342 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:33.573318 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") pod \"maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:33.675657 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:33.675623 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:33.816032 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:33.816006 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp"] Jul 22 04:56:33.817777 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:56:33.817746 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd392003a_2edc_4fe7_a8f1_469095b0d3ea.slice/crio-34b8b2bc56afeb9cf8f1bf0c72bf07cffc31bd93b282369fa2ef655dd352736e WatchSource:0}: Error finding container 34b8b2bc56afeb9cf8f1bf0c72bf07cffc31bd93b282369fa2ef655dd352736e: Status 404 returned error can't find the container with id 34b8b2bc56afeb9cf8f1bf0c72bf07cffc31bd93b282369fa2ef655dd352736e Jul 22 04:56:34.615355 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:34.615265 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" event={"ID":"d392003a-2edc-4fe7-a8f1-469095b0d3ea","Type":"ContainerStarted","Data":"85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e"} Jul 22 04:56:34.615355 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:34.615321 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" event={"ID":"d392003a-2edc-4fe7-a8f1-469095b0d3ea","Type":"ContainerStarted","Data":"34b8b2bc56afeb9cf8f1bf0c72bf07cffc31bd93b282369fa2ef655dd352736e"} Jul 22 04:56:34.615749 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:34.615458 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:56:34.631944 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:34.631891 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" podStartSLOduration=3.187137447 podStartE2EDuration="3.631872793s" totalImagesPullingTime="444.735346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:56:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:56:34.629742699 +0000 UTC m=+1918.719021889" watchObservedRunningTime="2026-07-22 04:56:34.631872793 +0000 UTC m=+1918.721151965" Jul 22 04:56:40.626880 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:56:40.626851 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:57:07.907497 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:07.907459 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp"] Jul 22 04:57:07.908118 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:07.907666 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" podUID="d392003a-2edc-4fe7-a8f1-469095b0d3ea" containerName="maas-api" containerID="cri-o://85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e" gracePeriod=30 Jul 22 04:57:08.163127 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.163050 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:57:08.278977 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.278941 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d392003a-2edc-4fe7-a8f1-469095b0d3ea-openshift-service-ca\") pod \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " Jul 22 04:57:08.279187 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.279009 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") pod \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " Jul 22 04:57:08.279187 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.279130 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vsp2\" (UniqueName: \"kubernetes.io/projected/d392003a-2edc-4fe7-a8f1-469095b0d3ea-kube-api-access-4vsp2\") pod \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\" (UID: \"d392003a-2edc-4fe7-a8f1-469095b0d3ea\") " Jul 22 04:57:08.279474 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.279435 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d392003a-2edc-4fe7-a8f1-469095b0d3ea-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d392003a-2edc-4fe7-a8f1-469095b0d3ea" (UID: "d392003a-2edc-4fe7-a8f1-469095b0d3ea"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:57:08.281202 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.281172 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d392003a-2edc-4fe7-a8f1-469095b0d3ea" (UID: "d392003a-2edc-4fe7-a8f1-469095b0d3ea"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:57:08.281301 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.281263 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d392003a-2edc-4fe7-a8f1-469095b0d3ea-kube-api-access-4vsp2" (OuterVolumeSpecName: "kube-api-access-4vsp2") pod "d392003a-2edc-4fe7-a8f1-469095b0d3ea" (UID: "d392003a-2edc-4fe7-a8f1-469095b0d3ea"). InnerVolumeSpecName "kube-api-access-4vsp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:57:08.380559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.380508 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vsp2\" (UniqueName: \"kubernetes.io/projected/d392003a-2edc-4fe7-a8f1-469095b0d3ea-kube-api-access-4vsp2\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:57:08.380559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.380555 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d392003a-2edc-4fe7-a8f1-469095b0d3ea-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:57:08.380559 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.380566 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d392003a-2edc-4fe7-a8f1-469095b0d3ea-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:57:08.810491 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.810452 2566 generic.go:356] "Generic (PLEG): container finished" podID="d392003a-2edc-4fe7-a8f1-469095b0d3ea" containerID="85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e" exitCode=0 Jul 22 04:57:08.810675 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.810503 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" Jul 22 04:57:08.810675 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.810507 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" event={"ID":"d392003a-2edc-4fe7-a8f1-469095b0d3ea","Type":"ContainerDied","Data":"85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e"} Jul 22 04:57:08.810675 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.810561 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp" event={"ID":"d392003a-2edc-4fe7-a8f1-469095b0d3ea","Type":"ContainerDied","Data":"34b8b2bc56afeb9cf8f1bf0c72bf07cffc31bd93b282369fa2ef655dd352736e"} Jul 22 04:57:08.810675 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.810590 2566 scope.go:117] "RemoveContainer" containerID="85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e" Jul 22 04:57:08.819990 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.819973 2566 scope.go:117] "RemoveContainer" containerID="85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e" Jul 22 04:57:08.820292 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:57:08.820270 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e\": container with ID starting with 85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e not found: ID does not exist" containerID="85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e" Jul 22 04:57:08.820374 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.820300 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e"} err="failed to get container status \"85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e\": rpc error: code = NotFound desc = could not find container \"85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e\": container with ID starting with 85731e715670fcaf67f81e6eae3e2d49ccd5c38009d71cce0bba2619d8eae83e not found: ID does not exist" Jul 22 04:57:08.831578 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.831551 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp"] Jul 22 04:57:08.836918 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.836890 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e7529972-6d765f6f97-9r7fp"] Jul 22 04:57:08.931661 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:08.931625 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d392003a-2edc-4fe7-a8f1-469095b0d3ea" path="/var/lib/kubelet/pods/d392003a-2edc-4fe7-a8f1-469095b0d3ea/volumes" Jul 22 04:57:16.513119 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.513056 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m"] Jul 22 04:57:16.513729 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.513677 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d392003a-2edc-4fe7-a8f1-469095b0d3ea" containerName="maas-api" Jul 22 04:57:16.513729 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.513693 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d392003a-2edc-4fe7-a8f1-469095b0d3ea" containerName="maas-api" Jul 22 04:57:16.513874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.513804 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d392003a-2edc-4fe7-a8f1-469095b0d3ea" containerName="maas-api" Jul 22 04:57:16.518858 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.518838 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.521388 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.521364 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7a6c5591-serving-cert\"" Jul 22 04:57:16.528714 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.528688 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m"] Jul 22 04:57:16.655695 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.655659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/693b3794-ca78-4eff-8f4f-199c70496759-maas-api-tls\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.655695 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.655697 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6czqx\" (UniqueName: \"kubernetes.io/projected/693b3794-ca78-4eff-8f4f-199c70496759-kube-api-access-6czqx\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.655925 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.655722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/693b3794-ca78-4eff-8f4f-199c70496759-openshift-service-ca\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.757320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.757280 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/693b3794-ca78-4eff-8f4f-199c70496759-maas-api-tls\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.757320 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.757325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6czqx\" (UniqueName: \"kubernetes.io/projected/693b3794-ca78-4eff-8f4f-199c70496759-kube-api-access-6czqx\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.757542 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.757349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/693b3794-ca78-4eff-8f4f-199c70496759-openshift-service-ca\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.757981 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.757955 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/693b3794-ca78-4eff-8f4f-199c70496759-openshift-service-ca\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.759875 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.759850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/693b3794-ca78-4eff-8f4f-199c70496759-maas-api-tls\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.764710 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.764648 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6czqx\" (UniqueName: \"kubernetes.io/projected/693b3794-ca78-4eff-8f4f-199c70496759-kube-api-access-6czqx\") pod \"maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.831428 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.831396 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:16.973095 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:16.973037 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m"] Jul 22 04:57:16.975399 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:57:16.975371 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod693b3794_ca78_4eff_8f4f_199c70496759.slice/crio-a7801fe47ae0ca7e37d17f109975b85e084f6b0272143a354341bc253de6c6bd WatchSource:0}: Error finding container a7801fe47ae0ca7e37d17f109975b85e084f6b0272143a354341bc253de6c6bd: Status 404 returned error can't find the container with id a7801fe47ae0ca7e37d17f109975b85e084f6b0272143a354341bc253de6c6bd Jul 22 04:57:17.863607 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:17.863561 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" event={"ID":"693b3794-ca78-4eff-8f4f-199c70496759","Type":"ContainerStarted","Data":"edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517"} Jul 22 04:57:17.864141 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:17.863662 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:17.864141 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:17.863681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" event={"ID":"693b3794-ca78-4eff-8f4f-199c70496759","Type":"ContainerStarted","Data":"a7801fe47ae0ca7e37d17f109975b85e084f6b0272143a354341bc253de6c6bd"} Jul 22 04:57:17.881154 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:17.881096 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" podStartSLOduration=1.4908021169999999 podStartE2EDuration="1.881064564s" totalImagesPullingTime="390.262447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:57:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:57:17.878197804 +0000 UTC m=+1961.967476991" watchObservedRunningTime="2026-07-22 04:57:17.881064564 +0000 UTC m=+1961.970343736" Jul 22 04:57:23.876622 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:23.876591 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:51.614709 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.614674 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m"] Jul 22 04:57:51.615220 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.614889 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" podUID="693b3794-ca78-4eff-8f4f-199c70496759" containerName="maas-api" containerID="cri-o://edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517" gracePeriod=30 Jul 22 04:57:51.864871 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.864810 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:51.974617 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.974589 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6czqx\" (UniqueName: \"kubernetes.io/projected/693b3794-ca78-4eff-8f4f-199c70496759-kube-api-access-6czqx\") pod \"693b3794-ca78-4eff-8f4f-199c70496759\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " Jul 22 04:57:51.974617 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.974621 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/693b3794-ca78-4eff-8f4f-199c70496759-openshift-service-ca\") pod \"693b3794-ca78-4eff-8f4f-199c70496759\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " Jul 22 04:57:51.974874 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.974662 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/693b3794-ca78-4eff-8f4f-199c70496759-maas-api-tls\") pod \"693b3794-ca78-4eff-8f4f-199c70496759\" (UID: \"693b3794-ca78-4eff-8f4f-199c70496759\") " Jul 22 04:57:51.975047 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.975014 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/693b3794-ca78-4eff-8f4f-199c70496759-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "693b3794-ca78-4eff-8f4f-199c70496759" (UID: "693b3794-ca78-4eff-8f4f-199c70496759"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:57:51.976836 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.976810 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/693b3794-ca78-4eff-8f4f-199c70496759-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "693b3794-ca78-4eff-8f4f-199c70496759" (UID: "693b3794-ca78-4eff-8f4f-199c70496759"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:57:51.976931 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:51.976859 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/693b3794-ca78-4eff-8f4f-199c70496759-kube-api-access-6czqx" (OuterVolumeSpecName: "kube-api-access-6czqx") pod "693b3794-ca78-4eff-8f4f-199c70496759" (UID: "693b3794-ca78-4eff-8f4f-199c70496759"). InnerVolumeSpecName "kube-api-access-6czqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:57:52.051873 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.051840 2566 generic.go:356] "Generic (PLEG): container finished" podID="693b3794-ca78-4eff-8f4f-199c70496759" containerID="edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517" exitCode=0 Jul 22 04:57:52.052121 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.051892 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" Jul 22 04:57:52.052121 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.051931 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" event={"ID":"693b3794-ca78-4eff-8f4f-199c70496759","Type":"ContainerDied","Data":"edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517"} Jul 22 04:57:52.052121 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.051991 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m" event={"ID":"693b3794-ca78-4eff-8f4f-199c70496759","Type":"ContainerDied","Data":"a7801fe47ae0ca7e37d17f109975b85e084f6b0272143a354341bc253de6c6bd"} Jul 22 04:57:52.052121 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.052024 2566 scope.go:117] "RemoveContainer" containerID="edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517" Jul 22 04:57:52.061715 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.061700 2566 scope.go:117] "RemoveContainer" containerID="edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517" Jul 22 04:57:52.061956 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:57:52.061940 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517\": container with ID starting with edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517 not found: ID does not exist" containerID="edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517" Jul 22 04:57:52.061995 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.061966 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517"} err="failed to get container status \"edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517\": rpc error: code = NotFound desc = could not find container \"edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517\": container with ID starting with edcb60990be2a9032827ebb2e165659d558a14cdf4d779cde66b9d3b1da2f517 not found: ID does not exist" Jul 22 04:57:52.075811 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.075793 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/693b3794-ca78-4eff-8f4f-199c70496759-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:57:52.075885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.075813 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6czqx\" (UniqueName: \"kubernetes.io/projected/693b3794-ca78-4eff-8f4f-199c70496759-kube-api-access-6czqx\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:57:52.075885 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.075823 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/693b3794-ca78-4eff-8f4f-199c70496759-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:57:52.081680 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.081658 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m"] Jul 22 04:57:52.087062 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.087042 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7a6c5591-5c7498dd7c-xkg9m"] Jul 22 04:57:52.931112 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:57:52.931060 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="693b3794-ca78-4eff-8f4f-199c70496759" path="/var/lib/kubelet/pods/693b3794-ca78-4eff-8f4f-199c70496759/volumes" Jul 22 04:59:34.223380 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.223293 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p"] Jul 22 04:59:34.223971 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.223745 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="693b3794-ca78-4eff-8f4f-199c70496759" containerName="maas-api" Jul 22 04:59:34.223971 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.223764 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="693b3794-ca78-4eff-8f4f-199c70496759" containerName="maas-api" Jul 22 04:59:34.223971 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.223835 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="693b3794-ca78-4eff-8f4f-199c70496759" containerName="maas-api" Jul 22 04:59:34.229248 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.229226 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.231489 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.231467 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0c0adb03-serving-cert\"" Jul 22 04:59:34.241043 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.241012 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p"] Jul 22 04:59:34.282368 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.282331 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2xxj\" (UniqueName: \"kubernetes.io/projected/29ec2e86-a755-4187-858c-3f25c6e22b4c-kube-api-access-d2xxj\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.282527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.282387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.282527 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.282477 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ec2e86-a755-4187-858c-3f25c6e22b4c-openshift-service-ca\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.383797 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.383753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ec2e86-a755-4187-858c-3f25c6e22b4c-openshift-service-ca\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.383970 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.383817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2xxj\" (UniqueName: \"kubernetes.io/projected/29ec2e86-a755-4187-858c-3f25c6e22b4c-kube-api-access-d2xxj\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.383970 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.383868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.384045 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:59:34.384009 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-serving-cert: secret "maas-api-e2e-mt-0c0adb03-serving-cert" not found Jul 22 04:59:34.384147 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:59:34.384136 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls podName:29ec2e86-a755-4187-858c-3f25c6e22b4c nodeName:}" failed. No retries permitted until 2026-07-22 04:59:34.884114237 +0000 UTC m=+2098.973393388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls") pod "maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" (UID: "29ec2e86-a755-4187-858c-3f25c6e22b4c") : secret "maas-api-e2e-mt-0c0adb03-serving-cert" not found Jul 22 04:59:34.384459 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.384440 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ec2e86-a755-4187-858c-3f25c6e22b4c-openshift-service-ca\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.396484 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.396455 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2xxj\" (UniqueName: \"kubernetes.io/projected/29ec2e86-a755-4187-858c-3f25c6e22b4c-kube-api-access-d2xxj\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.888356 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.888320 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:34.890744 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:34.890718 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls\") pod \"maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:35.142822 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:35.142792 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:35.277887 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:35.277854 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p"] Jul 22 04:59:35.279226 ip-10-0-142-182 kubenswrapper[2566]: W0722 04:59:35.279196 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ec2e86_a755_4187_858c_3f25c6e22b4c.slice/crio-c39bb32a184c0655147dd4576b64159fe6541122daac0ee7df64e897c5c68197 WatchSource:0}: Error finding container c39bb32a184c0655147dd4576b64159fe6541122daac0ee7df64e897c5c68197: Status 404 returned error can't find the container with id c39bb32a184c0655147dd4576b64159fe6541122daac0ee7df64e897c5c68197 Jul 22 04:59:35.622522 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:35.622435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" event={"ID":"29ec2e86-a755-4187-858c-3f25c6e22b4c","Type":"ContainerStarted","Data":"c39bb32a184c0655147dd4576b64159fe6541122daac0ee7df64e897c5c68197"} Jul 22 04:59:36.629916 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:36.629878 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" event={"ID":"29ec2e86-a755-4187-858c-3f25c6e22b4c","Type":"ContainerStarted","Data":"e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289"} Jul 22 04:59:36.630330 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:36.629988 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:36.648493 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:36.648446 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" podStartSLOduration=2.249336439 podStartE2EDuration="2.648431053s" totalImagesPullingTime="399.094614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:59:36.646580879 +0000 UTC m=+2100.735860096" watchObservedRunningTime="2026-07-22 04:59:36.648431053 +0000 UTC m=+2100.737710233" Jul 22 04:59:42.641705 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:42.641673 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:56.335225 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.335190 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p"] Jul 22 04:59:56.335751 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.335437 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" podUID="29ec2e86-a755-4187-858c-3f25c6e22b4c" containerName="maas-api" containerID="cri-o://e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289" gracePeriod=30 Jul 22 04:59:56.584655 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.584628 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:56.585714 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.585675 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2xxj\" (UniqueName: \"kubernetes.io/projected/29ec2e86-a755-4187-858c-3f25c6e22b4c-kube-api-access-d2xxj\") pod \"29ec2e86-a755-4187-858c-3f25c6e22b4c\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " Jul 22 04:59:56.585769 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.585724 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls\") pod \"29ec2e86-a755-4187-858c-3f25c6e22b4c\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " Jul 22 04:59:56.585813 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.585799 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ec2e86-a755-4187-858c-3f25c6e22b4c-openshift-service-ca\") pod \"29ec2e86-a755-4187-858c-3f25c6e22b4c\" (UID: \"29ec2e86-a755-4187-858c-3f25c6e22b4c\") " Jul 22 04:59:56.586176 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.586152 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ec2e86-a755-4187-858c-3f25c6e22b4c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "29ec2e86-a755-4187-858c-3f25c6e22b4c" (UID: "29ec2e86-a755-4187-858c-3f25c6e22b4c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:59:56.587773 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.587751 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "29ec2e86-a755-4187-858c-3f25c6e22b4c" (UID: "29ec2e86-a755-4187-858c-3f25c6e22b4c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 04:59:56.587917 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.587894 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ec2e86-a755-4187-858c-3f25c6e22b4c-kube-api-access-d2xxj" (OuterVolumeSpecName: "kube-api-access-d2xxj") pod "29ec2e86-a755-4187-858c-3f25c6e22b4c" (UID: "29ec2e86-a755-4187-858c-3f25c6e22b4c"). InnerVolumeSpecName "kube-api-access-d2xxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:59:56.686699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.686665 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/29ec2e86-a755-4187-858c-3f25c6e22b4c-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:59:56.686699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.686693 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2xxj\" (UniqueName: \"kubernetes.io/projected/29ec2e86-a755-4187-858c-3f25c6e22b4c-kube-api-access-d2xxj\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:59:56.686699 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.686703 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/29ec2e86-a755-4187-858c-3f25c6e22b4c-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 04:59:56.740708 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.740675 2566 generic.go:356] "Generic (PLEG): container finished" podID="29ec2e86-a755-4187-858c-3f25c6e22b4c" containerID="e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289" exitCode=0 Jul 22 04:59:56.740891 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.740726 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" Jul 22 04:59:56.740891 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.740752 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" event={"ID":"29ec2e86-a755-4187-858c-3f25c6e22b4c","Type":"ContainerDied","Data":"e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289"} Jul 22 04:59:56.740891 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.740814 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p" event={"ID":"29ec2e86-a755-4187-858c-3f25c6e22b4c","Type":"ContainerDied","Data":"c39bb32a184c0655147dd4576b64159fe6541122daac0ee7df64e897c5c68197"} Jul 22 04:59:56.740891 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.740847 2566 scope.go:117] "RemoveContainer" containerID="e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289" Jul 22 04:59:56.750668 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.750649 2566 scope.go:117] "RemoveContainer" containerID="e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289" Jul 22 04:59:56.750954 ip-10-0-142-182 kubenswrapper[2566]: E0722 04:59:56.750937 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289\": container with ID starting with e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289 not found: ID does not exist" containerID="e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289" Jul 22 04:59:56.751001 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.750962 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289"} err="failed to get container status \"e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289\": rpc error: code = NotFound desc = could not find container \"e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289\": container with ID starting with e82d6c87f1052aa94ebc7890e94803a77ff2d6da182aa17212bc16b04a564289 not found: ID does not exist" Jul 22 04:59:56.762346 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.762322 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p"] Jul 22 04:59:56.765482 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.765461 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0c0adb03-bc5c975d9-wv79p"] Jul 22 04:59:56.931791 ip-10-0-142-182 kubenswrapper[2566]: I0722 04:59:56.931760 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ec2e86-a755-4187-858c-3f25c6e22b4c" path="/var/lib/kubelet/pods/29ec2e86-a755-4187-858c-3f25c6e22b4c/volumes" Jul 22 05:00:29.505649 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.505563 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl"] Jul 22 05:00:29.506202 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.506186 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ec2e86-a755-4187-858c-3f25c6e22b4c" containerName="maas-api" Jul 22 05:00:29.506250 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.506206 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ec2e86-a755-4187-858c-3f25c6e22b4c" containerName="maas-api" Jul 22 05:00:29.506333 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.506323 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ec2e86-a755-4187-858c-3f25c6e22b4c" containerName="maas-api" Jul 22 05:00:29.509836 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.509815 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.512305 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.512282 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-d559fe69-serving-cert\"" Jul 22 05:00:29.518180 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.518141 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl"] Jul 22 05:00:29.587639 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.587599 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fa54b32-1693-423b-8ec4-6b4a73f59876-openshift-service-ca\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.587837 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.587659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.587837 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.587695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t74fq\" (UniqueName: \"kubernetes.io/projected/4fa54b32-1693-423b-8ec4-6b4a73f59876-kube-api-access-t74fq\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.688745 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.688707 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.688745 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.688758 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t74fq\" (UniqueName: \"kubernetes.io/projected/4fa54b32-1693-423b-8ec4-6b4a73f59876-kube-api-access-t74fq\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.688970 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.688821 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fa54b32-1693-423b-8ec4-6b4a73f59876-openshift-service-ca\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.688970 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:00:29.688865 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-serving-cert: secret "maas-api-e2e-mt-d559fe69-serving-cert" not found Jul 22 05:00:29.688970 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:00:29.688946 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls podName:4fa54b32-1693-423b-8ec4-6b4a73f59876 nodeName:}" failed. No retries permitted until 2026-07-22 05:00:30.188925442 +0000 UTC m=+2154.278204595 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls") pod "maas-api-e2e-mt-d559fe69-78f746486f-j65fl" (UID: "4fa54b32-1693-423b-8ec4-6b4a73f59876") : secret "maas-api-e2e-mt-d559fe69-serving-cert" not found Jul 22 05:00:29.689423 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.689407 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fa54b32-1693-423b-8ec4-6b4a73f59876-openshift-service-ca\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:29.697816 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:29.697789 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t74fq\" (UniqueName: \"kubernetes.io/projected/4fa54b32-1693-423b-8ec4-6b4a73f59876-kube-api-access-t74fq\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:30.194291 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:30.194249 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:30.196784 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:30.196755 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls\") pod \"maas-api-e2e-mt-d559fe69-78f746486f-j65fl\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:30.422367 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:30.422324 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:30.581388 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:30.581361 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl"] Jul 22 05:00:30.583293 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:00:30.583263 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa54b32_1693_423b_8ec4_6b4a73f59876.slice/crio-bec61245ba38ee36041724cb732b0f54452d61ec9a17fea38ad4729caefd3859 WatchSource:0}: Error finding container bec61245ba38ee36041724cb732b0f54452d61ec9a17fea38ad4729caefd3859: Status 404 returned error can't find the container with id bec61245ba38ee36041724cb732b0f54452d61ec9a17fea38ad4729caefd3859 Jul 22 05:00:30.584645 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:30.584629 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 05:00:30.933777 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:30.933722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" event={"ID":"4fa54b32-1693-423b-8ec4-6b4a73f59876","Type":"ContainerStarted","Data":"bec61245ba38ee36041724cb732b0f54452d61ec9a17fea38ad4729caefd3859"} Jul 22 05:00:31.943447 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:31.943400 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" event={"ID":"4fa54b32-1693-423b-8ec4-6b4a73f59876","Type":"ContainerStarted","Data":"e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b"} Jul 22 05:00:31.943880 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:31.943579 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:00:31.959327 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:31.959268 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" podStartSLOduration=2.515269397 podStartE2EDuration="2.959250364s" totalImagesPullingTime="443.980967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:00:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:00:31.958446693 +0000 UTC m=+2156.047725891" watchObservedRunningTime="2026-07-22 05:00:31.959250364 +0000 UTC m=+2156.048529548" Jul 22 05:00:37.957908 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:00:37.957878 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:01:22.492339 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.492306 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl"] Jul 22 05:01:22.494880 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.492571 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" podUID="4fa54b32-1693-423b-8ec4-6b4a73f59876" containerName="maas-api" containerID="cri-o://e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b" gracePeriod=30 Jul 22 05:01:22.734832 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.734803 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:01:22.889516 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.889433 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t74fq\" (UniqueName: \"kubernetes.io/projected/4fa54b32-1693-423b-8ec4-6b4a73f59876-kube-api-access-t74fq\") pod \"4fa54b32-1693-423b-8ec4-6b4a73f59876\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " Jul 22 05:01:22.889516 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.889492 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fa54b32-1693-423b-8ec4-6b4a73f59876-openshift-service-ca\") pod \"4fa54b32-1693-423b-8ec4-6b4a73f59876\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " Jul 22 05:01:22.889516 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.889513 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls\") pod \"4fa54b32-1693-423b-8ec4-6b4a73f59876\" (UID: \"4fa54b32-1693-423b-8ec4-6b4a73f59876\") " Jul 22 05:01:22.889874 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.889849 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fa54b32-1693-423b-8ec4-6b4a73f59876-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4fa54b32-1693-423b-8ec4-6b4a73f59876" (UID: "4fa54b32-1693-423b-8ec4-6b4a73f59876"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:01:22.891547 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.891519 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4fa54b32-1693-423b-8ec4-6b4a73f59876" (UID: "4fa54b32-1693-423b-8ec4-6b4a73f59876"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 05:01:22.891726 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.891699 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fa54b32-1693-423b-8ec4-6b4a73f59876-kube-api-access-t74fq" (OuterVolumeSpecName: "kube-api-access-t74fq") pod "4fa54b32-1693-423b-8ec4-6b4a73f59876" (UID: "4fa54b32-1693-423b-8ec4-6b4a73f59876"). InnerVolumeSpecName "kube-api-access-t74fq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:01:22.990989 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.990954 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t74fq\" (UniqueName: \"kubernetes.io/projected/4fa54b32-1693-423b-8ec4-6b4a73f59876-kube-api-access-t74fq\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:01:22.990989 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.990985 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fa54b32-1693-423b-8ec4-6b4a73f59876-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:01:22.990989 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:22.990995 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4fa54b32-1693-423b-8ec4-6b4a73f59876-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:01:23.238265 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.238227 2566 generic.go:356] "Generic (PLEG): container finished" podID="4fa54b32-1693-423b-8ec4-6b4a73f59876" containerID="e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b" exitCode=0 Jul 22 05:01:23.238265 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.238269 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" Jul 22 05:01:23.238472 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.238309 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" event={"ID":"4fa54b32-1693-423b-8ec4-6b4a73f59876","Type":"ContainerDied","Data":"e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b"} Jul 22 05:01:23.238472 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.238356 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl" event={"ID":"4fa54b32-1693-423b-8ec4-6b4a73f59876","Type":"ContainerDied","Data":"bec61245ba38ee36041724cb732b0f54452d61ec9a17fea38ad4729caefd3859"} Jul 22 05:01:23.238472 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.238380 2566 scope.go:117] "RemoveContainer" containerID="e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b" Jul 22 05:01:23.247520 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.247502 2566 scope.go:117] "RemoveContainer" containerID="e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b" Jul 22 05:01:23.247782 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:01:23.247750 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b\": container with ID starting with e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b not found: ID does not exist" containerID="e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b" Jul 22 05:01:23.247888 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.247787 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b"} err="failed to get container status \"e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b\": rpc error: code = NotFound desc = could not find container \"e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b\": container with ID starting with e98b9a7c9d5be04a8add05ffdb377bca900e2dcfd7b5fb8970470a942f482f6b not found: ID does not exist" Jul 22 05:01:23.255769 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.255748 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl"] Jul 22 05:01:23.259465 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:23.259445 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-d559fe69-78f746486f-j65fl"] Jul 22 05:01:24.932014 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:01:24.931968 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fa54b32-1693-423b-8ec4-6b4a73f59876" path="/var/lib/kubelet/pods/4fa54b32-1693-423b-8ec4-6b4a73f59876/volumes" Jul 22 05:03:29.419418 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.419317 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm"] Jul 22 05:03:29.420165 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.419899 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fa54b32-1693-423b-8ec4-6b4a73f59876" containerName="maas-api" Jul 22 05:03:29.420165 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.419917 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fa54b32-1693-423b-8ec4-6b4a73f59876" containerName="maas-api" Jul 22 05:03:29.420165 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.420010 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fa54b32-1693-423b-8ec4-6b4a73f59876" containerName="maas-api" Jul 22 05:03:29.423227 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.423210 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.425455 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.425434 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-e9e018-serving-cert\"" Jul 22 05:03:29.440736 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.440702 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm"] Jul 22 05:03:29.515976 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.515939 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1319fc-204a-46e0-9ea6-b4a652d88a09-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.515976 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.515977 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.516216 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.516092 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25tg2\" (UniqueName: \"kubernetes.io/projected/0f1319fc-204a-46e0-9ea6-b4a652d88a09-kube-api-access-25tg2\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.616970 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.616927 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1319fc-204a-46e0-9ea6-b4a652d88a09-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.617166 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.616975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.617166 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.617037 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25tg2\" (UniqueName: \"kubernetes.io/projected/0f1319fc-204a-46e0-9ea6-b4a652d88a09-kube-api-access-25tg2\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.617297 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:03:29.617186 2566 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-serving-cert: secret "maas-api-e2e-inf-a-e9e018-serving-cert" not found Jul 22 05:03:29.617297 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:03:29.617262 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls podName:0f1319fc-204a-46e0-9ea6-b4a652d88a09 nodeName:}" failed. No retries permitted until 2026-07-22 05:03:30.117242544 +0000 UTC m=+2334.206521699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls") pod "maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" (UID: "0f1319fc-204a-46e0-9ea6-b4a652d88a09") : secret "maas-api-e2e-inf-a-e9e018-serving-cert" not found Jul 22 05:03:29.617741 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.617711 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1319fc-204a-46e0-9ea6-b4a652d88a09-openshift-service-ca\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:29.628739 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:29.628713 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25tg2\" (UniqueName: \"kubernetes.io/projected/0f1319fc-204a-46e0-9ea6-b4a652d88a09-kube-api-access-25tg2\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:30.122543 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:30.122503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:30.124901 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:30.124883 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls\") pod \"maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:30.333695 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:30.333652 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:30.472044 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:30.472009 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm"] Jul 22 05:03:30.472917 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:03:30.472891 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1319fc_204a_46e0_9ea6_b4a652d88a09.slice/crio-dd1cd5e99fb05e4e8ef99e1b977efab1f38239888073cbc57e111c09bd0ba23b WatchSource:0}: Error finding container dd1cd5e99fb05e4e8ef99e1b977efab1f38239888073cbc57e111c09bd0ba23b: Status 404 returned error can't find the container with id dd1cd5e99fb05e4e8ef99e1b977efab1f38239888073cbc57e111c09bd0ba23b Jul 22 05:03:30.930724 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:30.930685 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" event={"ID":"0f1319fc-204a-46e0-9ea6-b4a652d88a09","Type":"ContainerStarted","Data":"dd1cd5e99fb05e4e8ef99e1b977efab1f38239888073cbc57e111c09bd0ba23b"} Jul 22 05:03:31.932007 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:31.931971 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" event={"ID":"0f1319fc-204a-46e0-9ea6-b4a652d88a09","Type":"ContainerStarted","Data":"a1636c4b58e2e3c431de1d72a76768825e6a73dd01ef08c87b376a53c99b4520"} Jul 22 05:03:31.932430 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:31.932110 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:03:31.947838 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:31.947798 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" podStartSLOduration=2.469116021 podStartE2EDuration="2.947784142s" totalImagesPullingTime="478.668121ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 05:03:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:03:31.946885963 +0000 UTC m=+2336.036165132" watchObservedRunningTime="2026-07-22 05:03:31.947784142 +0000 UTC m=+2336.037063314" Jul 22 05:03:37.943479 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:03:37.943450 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:05:23.406535 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.406436 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm"] Jul 22 05:05:23.407136 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.406668 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" podUID="0f1319fc-204a-46e0-9ea6-b4a652d88a09" containerName="maas-api" containerID="cri-o://a1636c4b58e2e3c431de1d72a76768825e6a73dd01ef08c87b376a53c99b4520" gracePeriod=30 Jul 22 05:05:23.558753 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.558718 2566 generic.go:356] "Generic (PLEG): container finished" podID="0f1319fc-204a-46e0-9ea6-b4a652d88a09" containerID="a1636c4b58e2e3c431de1d72a76768825e6a73dd01ef08c87b376a53c99b4520" exitCode=0 Jul 22 05:05:23.558962 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.558769 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" event={"ID":"0f1319fc-204a-46e0-9ea6-b4a652d88a09","Type":"ContainerDied","Data":"a1636c4b58e2e3c431de1d72a76768825e6a73dd01ef08c87b376a53c99b4520"} Jul 22 05:05:23.662290 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.662230 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:05:23.768210 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.768178 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1319fc-204a-46e0-9ea6-b4a652d88a09-openshift-service-ca\") pod \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " Jul 22 05:05:23.768420 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.768289 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls\") pod \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " Jul 22 05:05:23.768420 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.768328 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25tg2\" (UniqueName: \"kubernetes.io/projected/0f1319fc-204a-46e0-9ea6-b4a652d88a09-kube-api-access-25tg2\") pod \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\" (UID: \"0f1319fc-204a-46e0-9ea6-b4a652d88a09\") " Jul 22 05:05:23.768620 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.768596 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f1319fc-204a-46e0-9ea6-b4a652d88a09-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0f1319fc-204a-46e0-9ea6-b4a652d88a09" (UID: "0f1319fc-204a-46e0-9ea6-b4a652d88a09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 05:05:23.770338 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.770310 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0f1319fc-204a-46e0-9ea6-b4a652d88a09" (UID: "0f1319fc-204a-46e0-9ea6-b4a652d88a09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 05:05:23.770455 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.770430 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f1319fc-204a-46e0-9ea6-b4a652d88a09-kube-api-access-25tg2" (OuterVolumeSpecName: "kube-api-access-25tg2") pod "0f1319fc-204a-46e0-9ea6-b4a652d88a09" (UID: "0f1319fc-204a-46e0-9ea6-b4a652d88a09"). InnerVolumeSpecName "kube-api-access-25tg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:05:23.869306 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.869262 2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0f1319fc-204a-46e0-9ea6-b4a652d88a09-maas-api-tls\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:05:23.869306 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.869302 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25tg2\" (UniqueName: \"kubernetes.io/projected/0f1319fc-204a-46e0-9ea6-b4a652d88a09-kube-api-access-25tg2\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:05:23.869306 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:23.869312 2566 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0f1319fc-204a-46e0-9ea6-b4a652d88a09-openshift-service-ca\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:05:24.565627 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:24.565594 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" Jul 22 05:05:24.566223 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:24.565587 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm" event={"ID":"0f1319fc-204a-46e0-9ea6-b4a652d88a09","Type":"ContainerDied","Data":"dd1cd5e99fb05e4e8ef99e1b977efab1f38239888073cbc57e111c09bd0ba23b"} Jul 22 05:05:24.566223 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:24.565799 2566 scope.go:117] "RemoveContainer" containerID="a1636c4b58e2e3c431de1d72a76768825e6a73dd01ef08c87b376a53c99b4520" Jul 22 05:05:24.588238 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:24.588211 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm"] Jul 22 05:05:24.591866 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:24.591843 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-e9e018-7f6cf547db-mckmm"] Jul 22 05:05:24.936266 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:05:24.934200 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f1319fc-204a-46e0-9ea6-b4a652d88a09" path="/var/lib/kubelet/pods/0f1319fc-204a-46e0-9ea6-b4a652d88a09/volumes" Jul 22 05:13:03.414627 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.414586 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7629-139657504304688"] Jul 22 05:13:03.415286 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.415194 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f1319fc-204a-46e0-9ea6-b4a652d88a09" containerName="maas-api" Jul 22 05:13:03.415286 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.415217 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f1319fc-204a-46e0-9ea6-b4a652d88a09" containerName="maas-api" Jul 22 05:13:03.415422 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.415354 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f1319fc-204a-46e0-9ea6-b4a652d88a09" containerName="maas-api" Jul 22 05:13:03.418524 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.418504 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:03.422572 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.422530 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s4r5k\"" Jul 22 05:13:03.424746 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.424724 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504304688"] Jul 22 05:13:03.565266 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.565231 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsj6v\" (UniqueName: \"kubernetes.io/projected/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e-kube-api-access-bsj6v\") pod \"test-curl-7629-139657504304688\" (UID: \"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e\") " pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:03.666585 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.666495 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsj6v\" (UniqueName: \"kubernetes.io/projected/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e-kube-api-access-bsj6v\") pod \"test-curl-7629-139657504304688\" (UID: \"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e\") " pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:03.677028 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.677000 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsj6v\" (UniqueName: \"kubernetes.io/projected/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e-kube-api-access-bsj6v\") pod \"test-curl-7629-139657504304688\" (UID: \"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e\") " pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:03.729926 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.729893 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:03.874085 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.874039 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504304688"] Jul 22 05:13:03.875980 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:13:03.875935 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7adb0cf_9fcf_46aa_8b75_f6be40ddfe0e.slice/crio-0d2f8756bc8548e8dde3dd69d3b8267e055222c86e511e0a51ab7996843c4538 WatchSource:0}: Error finding container 0d2f8756bc8548e8dde3dd69d3b8267e055222c86e511e0a51ab7996843c4538: Status 404 returned error can't find the container with id 0d2f8756bc8548e8dde3dd69d3b8267e055222c86e511e0a51ab7996843c4538 Jul 22 05:13:03.877349 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:03.877330 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 05:13:04.097152 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:04.097050 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504304688" event={"ID":"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e","Type":"ContainerStarted","Data":"0d2f8756bc8548e8dde3dd69d3b8267e055222c86e511e0a51ab7996843c4538"} Jul 22 05:13:05.105130 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:05.105030 2566 generic.go:356] "Generic (PLEG): container finished" podID="d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e" containerID="5bd1d994a87a0cbb3487f16512003d3d87ce39095b2bf7f7141f21269d0eb874" exitCode=0 Jul 22 05:13:05.105484 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:05.105127 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504304688" event={"ID":"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e","Type":"ContainerDied","Data":"5bd1d994a87a0cbb3487f16512003d3d87ce39095b2bf7f7141f21269d0eb874"} Jul 22 05:13:06.244578 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.244552 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:06.282034 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.282009 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7629-139657504304688_d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e/test-curl-7629-139657504304688/0.log" Jul 22 05:13:06.287262 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.287242 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsj6v\" (UniqueName: \"kubernetes.io/projected/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e-kube-api-access-bsj6v\") pod \"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e\" (UID: \"d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e\") " Jul 22 05:13:06.289322 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.289293 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e-kube-api-access-bsj6v" (OuterVolumeSpecName: "kube-api-access-bsj6v") pod "d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e" (UID: "d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e"). InnerVolumeSpecName "kube-api-access-bsj6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:13:06.306962 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.306936 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7629-139657504304688"] Jul 22 05:13:06.311102 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.311057 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7629-139657504304688"] Jul 22 05:13:06.388231 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.388150 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsj6v\" (UniqueName: \"kubernetes.io/projected/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e-kube-api-access-bsj6v\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:13:06.435988 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.435956 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7629-139657504304944"] Jul 22 05:13:06.436399 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.436384 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e" containerName="test-curl-7629-139657504304688" Jul 22 05:13:06.436442 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.436400 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e" containerName="test-curl-7629-139657504304688" Jul 22 05:13:06.436482 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.436463 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e" containerName="test-curl-7629-139657504304688" Jul 22 05:13:06.439626 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.439610 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:06.447605 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.447586 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504304944"] Jul 22 05:13:06.489411 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.489378 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dvlz\" (UniqueName: \"kubernetes.io/projected/3d5a598e-7668-418d-9723-e0ef84f721be-kube-api-access-2dvlz\") pod \"test-curl-7629-139657504304944\" (UID: \"3d5a598e-7668-418d-9723-e0ef84f721be\") " pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:06.590134 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.590092 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dvlz\" (UniqueName: \"kubernetes.io/projected/3d5a598e-7668-418d-9723-e0ef84f721be-kube-api-access-2dvlz\") pod \"test-curl-7629-139657504304944\" (UID: \"3d5a598e-7668-418d-9723-e0ef84f721be\") " pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:06.600038 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.600001 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dvlz\" (UniqueName: \"kubernetes.io/projected/3d5a598e-7668-418d-9723-e0ef84f721be-kube-api-access-2dvlz\") pod \"test-curl-7629-139657504304944\" (UID: \"3d5a598e-7668-418d-9723-e0ef84f721be\") " pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:06.750696 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.750657 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:06.886186 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.886154 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504304944"] Jul 22 05:13:06.887827 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:13:06.887796 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d5a598e_7668_418d_9723_e0ef84f721be.slice/crio-3ac29d1ab78c07489c9e568a78642de4e86c03dad50a4f94d60079947602b9d1 WatchSource:0}: Error finding container 3ac29d1ab78c07489c9e568a78642de4e86c03dad50a4f94d60079947602b9d1: Status 404 returned error can't find the container with id 3ac29d1ab78c07489c9e568a78642de4e86c03dad50a4f94d60079947602b9d1 Jul 22 05:13:06.939612 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:06.939567 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e" path="/var/lib/kubelet/pods/d7adb0cf-9fcf-46aa-8b75-f6be40ddfe0e/volumes" Jul 22 05:13:07.117954 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:07.117857 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504304944" event={"ID":"3d5a598e-7668-418d-9723-e0ef84f721be","Type":"ContainerStarted","Data":"3ac29d1ab78c07489c9e568a78642de4e86c03dad50a4f94d60079947602b9d1"} Jul 22 05:13:07.119057 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:07.119037 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304688" Jul 22 05:13:07.119194 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:07.119137 2566 scope.go:117] "RemoveContainer" containerID="5bd1d994a87a0cbb3487f16512003d3d87ce39095b2bf7f7141f21269d0eb874" Jul 22 05:13:08.127498 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:08.127467 2566 generic.go:356] "Generic (PLEG): container finished" podID="3d5a598e-7668-418d-9723-e0ef84f721be" containerID="cf85610b6caa0b57cc092cc3e269767ffed330aa28cb89152c6c1b001fba17b6" exitCode=0 Jul 22 05:13:08.127916 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:08.127520 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504304944" event={"ID":"3d5a598e-7668-418d-9723-e0ef84f721be","Type":"ContainerDied","Data":"cf85610b6caa0b57cc092cc3e269767ffed330aa28cb89152c6c1b001fba17b6"} Jul 22 05:13:09.272808 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.272784 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:09.293671 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.293642 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7629-139657504304944_3d5a598e-7668-418d-9723-e0ef84f721be/test-curl-7629-139657504304944/0.log" Jul 22 05:13:09.311041 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.311011 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dvlz\" (UniqueName: \"kubernetes.io/projected/3d5a598e-7668-418d-9723-e0ef84f721be-kube-api-access-2dvlz\") pod \"3d5a598e-7668-418d-9723-e0ef84f721be\" (UID: \"3d5a598e-7668-418d-9723-e0ef84f721be\") " Jul 22 05:13:09.312978 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.312951 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d5a598e-7668-418d-9723-e0ef84f721be-kube-api-access-2dvlz" (OuterVolumeSpecName: "kube-api-access-2dvlz") pod "3d5a598e-7668-418d-9723-e0ef84f721be" (UID: "3d5a598e-7668-418d-9723-e0ef84f721be"). InnerVolumeSpecName "kube-api-access-2dvlz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:13:09.317430 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.317403 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7629-139657504304944"] Jul 22 05:13:09.323270 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.323250 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7629-139657504304944"] Jul 22 05:13:09.411850 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.411817 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dvlz\" (UniqueName: \"kubernetes.io/projected/3d5a598e-7668-418d-9723-e0ef84f721be-kube-api-access-2dvlz\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:13:09.477781 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.477745 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:09.478236 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.478217 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d5a598e-7668-418d-9723-e0ef84f721be" containerName="test-curl-7629-139657504304944" Jul 22 05:13:09.478346 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.478238 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d5a598e-7668-418d-9723-e0ef84f721be" containerName="test-curl-7629-139657504304944" Jul 22 05:13:09.478409 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.478382 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d5a598e-7668-418d-9723-e0ef84f721be" containerName="test-curl-7629-139657504304944" Jul 22 05:13:09.481442 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.481422 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:09.487604 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.487578 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:09.512569 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.512538 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhptw\" (UniqueName: \"kubernetes.io/projected/aa343d6f-4171-48a4-92cf-d5404662d9d7-kube-api-access-bhptw\") pod \"test-curl-7629-139657504404512\" (UID: \"aa343d6f-4171-48a4-92cf-d5404662d9d7\") " pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:09.613996 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.613957 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhptw\" (UniqueName: \"kubernetes.io/projected/aa343d6f-4171-48a4-92cf-d5404662d9d7-kube-api-access-bhptw\") pod \"test-curl-7629-139657504404512\" (UID: \"aa343d6f-4171-48a4-92cf-d5404662d9d7\") " pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:09.621903 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.621871 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhptw\" (UniqueName: \"kubernetes.io/projected/aa343d6f-4171-48a4-92cf-d5404662d9d7-kube-api-access-bhptw\") pod \"test-curl-7629-139657504404512\" (UID: \"aa343d6f-4171-48a4-92cf-d5404662d9d7\") " pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:09.792056 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.791972 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:09.927309 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:09.927280 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:09.928907 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:13:09.928874 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa343d6f_4171_48a4_92cf_d5404662d9d7.slice/crio-6a2a5162a4e59b652b5aa469e249470cc4d520713f293dd3a6db7d4336206dae WatchSource:0}: Error finding container 6a2a5162a4e59b652b5aa469e249470cc4d520713f293dd3a6db7d4336206dae: Status 404 returned error can't find the container with id 6a2a5162a4e59b652b5aa469e249470cc4d520713f293dd3a6db7d4336206dae Jul 22 05:13:10.140571 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:10.140470 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504404512" event={"ID":"aa343d6f-4171-48a4-92cf-d5404662d9d7","Type":"ContainerStarted","Data":"6a2a5162a4e59b652b5aa469e249470cc4d520713f293dd3a6db7d4336206dae"} Jul 22 05:13:10.141789 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:10.141771 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:10.141906 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:10.141808 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ac29d1ab78c07489c9e568a78642de4e86c03dad50a4f94d60079947602b9d1" Jul 22 05:13:10.152453 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:13:10.152424 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7629-139657504304944\" is forbidden: User \"system:node:ip-10-0-142-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-182.ec2.internal' and this object" podUID="3d5a598e-7668-418d-9723-e0ef84f721be" pod="opendatahub/test-curl-7629-139657504304944" Jul 22 05:13:10.933150 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:10.933108 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d5a598e-7668-418d-9723-e0ef84f721be" path="/var/lib/kubelet/pods/3d5a598e-7668-418d-9723-e0ef84f721be/volumes" Jul 22 05:13:11.149008 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:11.148974 2566 generic.go:356] "Generic (PLEG): container finished" podID="aa343d6f-4171-48a4-92cf-d5404662d9d7" containerID="94d9e8db670cc9ad0aeeeec908ecddbfdfd904c195933f33c66af38e7f690148" exitCode=0 Jul 22 05:13:11.149215 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:11.149063 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504404512" event={"ID":"aa343d6f-4171-48a4-92cf-d5404662d9d7","Type":"ContainerDied","Data":"94d9e8db670cc9ad0aeeeec908ecddbfdfd904c195933f33c66af38e7f690148"} Jul 22 05:13:12.294299 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.294271 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:12.314122 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.314088 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7629-139657504404512_aa343d6f-4171-48a4-92cf-d5404662d9d7/test-curl-7629-139657504404512/0.log" Jul 22 05:13:12.336850 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.336819 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:12.337012 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.336991 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhptw\" (UniqueName: \"kubernetes.io/projected/aa343d6f-4171-48a4-92cf-d5404662d9d7-kube-api-access-bhptw\") pod \"aa343d6f-4171-48a4-92cf-d5404662d9d7\" (UID: \"aa343d6f-4171-48a4-92cf-d5404662d9d7\") " Jul 22 05:13:12.339181 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.339149 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa343d6f-4171-48a4-92cf-d5404662d9d7-kube-api-access-bhptw" (OuterVolumeSpecName: "kube-api-access-bhptw") pod "aa343d6f-4171-48a4-92cf-d5404662d9d7" (UID: "aa343d6f-4171-48a4-92cf-d5404662d9d7"). InnerVolumeSpecName "kube-api-access-bhptw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:13:12.340055 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.340034 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:12.437802 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.437710 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhptw\" (UniqueName: \"kubernetes.io/projected/aa343d6f-4171-48a4-92cf-d5404662d9d7-kube-api-access-bhptw\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:13:12.479892 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.479856 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:12.480291 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.480278 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa343d6f-4171-48a4-92cf-d5404662d9d7" containerName="test-curl-7629-139657504404512" Jul 22 05:13:12.480339 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.480294 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa343d6f-4171-48a4-92cf-d5404662d9d7" containerName="test-curl-7629-139657504404512" Jul 22 05:13:12.480372 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.480358 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa343d6f-4171-48a4-92cf-d5404662d9d7" containerName="test-curl-7629-139657504404512" Jul 22 05:13:12.483479 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.483461 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:12.500704 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.500676 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:12.538439 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.538403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6tvk\" (UniqueName: \"kubernetes.io/projected/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd-kube-api-access-w6tvk\") pod \"test-curl-7629-139657504404512\" (UID: \"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd\") " pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:12.639591 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.639547 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6tvk\" (UniqueName: \"kubernetes.io/projected/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd-kube-api-access-w6tvk\") pod \"test-curl-7629-139657504404512\" (UID: \"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd\") " pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:12.648303 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.648271 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6tvk\" (UniqueName: \"kubernetes.io/projected/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd-kube-api-access-w6tvk\") pod \"test-curl-7629-139657504404512\" (UID: \"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd\") " pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:12.792375 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.792298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:12.932722 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.932691 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa343d6f-4171-48a4-92cf-d5404662d9d7" path="/var/lib/kubelet/pods/aa343d6f-4171-48a4-92cf-d5404662d9d7/volumes" Jul 22 05:13:12.933227 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:12.933205 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:12.934038 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:13:12.934015 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77f4693a_8cc8_4ab9_a1c6_5bc6fb3f43fd.slice/crio-b4ed64ca2f6aac02f26d9b06a2496ac86953753326c65a417a0c681f5cf7e8fb WatchSource:0}: Error finding container b4ed64ca2f6aac02f26d9b06a2496ac86953753326c65a417a0c681f5cf7e8fb: Status 404 returned error can't find the container with id b4ed64ca2f6aac02f26d9b06a2496ac86953753326c65a417a0c681f5cf7e8fb Jul 22 05:13:13.161339 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:13.161310 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:13.161541 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:13.161359 2566 scope.go:117] "RemoveContainer" containerID="94d9e8db670cc9ad0aeeeec908ecddbfdfd904c195933f33c66af38e7f690148" Jul 22 05:13:13.162673 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:13.162646 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504404512" event={"ID":"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd","Type":"ContainerStarted","Data":"b4ed64ca2f6aac02f26d9b06a2496ac86953753326c65a417a0c681f5cf7e8fb"} Jul 22 05:13:14.170681 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:14.170644 2566 generic.go:356] "Generic (PLEG): container finished" podID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" containerID="eeb9b239ac0461b64e30c18c8665c545e4ba84a8010cf7311dbadafa7db6bd3d" exitCode=0 Jul 22 05:13:14.171122 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:14.170699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504404512" event={"ID":"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd","Type":"ContainerDied","Data":"eeb9b239ac0461b64e30c18c8665c545e4ba84a8010cf7311dbadafa7db6bd3d"} Jul 22 05:13:15.314056 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.314032 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:15.335751 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.335720 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7629-139657504404512_77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd/test-curl-7629-139657504404512/0.log" Jul 22 05:13:15.358495 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.358468 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:15.362368 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.362345 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7629-139657504404512"] Jul 22 05:13:15.365180 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.365163 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6tvk\" (UniqueName: \"kubernetes.io/projected/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd-kube-api-access-w6tvk\") pod \"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd\" (UID: \"77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd\") " Jul 22 05:13:15.367209 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.367180 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd-kube-api-access-w6tvk" (OuterVolumeSpecName: "kube-api-access-w6tvk") pod "77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" (UID: "77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd"). InnerVolumeSpecName "kube-api-access-w6tvk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:13:15.466786 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.466693 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6tvk\" (UniqueName: \"kubernetes.io/projected/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd-kube-api-access-w6tvk\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:13:15.475335 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.475304 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:15.475743 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.475729 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" containerName="test-curl-7629-139657504404512" Jul 22 05:13:15.475783 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.475745 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" containerName="test-curl-7629-139657504404512" Jul 22 05:13:15.475824 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.475814 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" containerName="test-curl-7629-139657504404512" Jul 22 05:13:15.479041 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.479021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:15.482902 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.482878 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:15.567165 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.567131 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hksxg\" (UniqueName: \"kubernetes.io/projected/52a32494-2804-4187-9e84-942da2f02945-kube-api-access-hksxg\") pod \"test-curl-7629-139657504406960\" (UID: \"52a32494-2804-4187-9e84-942da2f02945\") " pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:15.668615 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.668577 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hksxg\" (UniqueName: \"kubernetes.io/projected/52a32494-2804-4187-9e84-942da2f02945-kube-api-access-hksxg\") pod \"test-curl-7629-139657504406960\" (UID: \"52a32494-2804-4187-9e84-942da2f02945\") " pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:15.676467 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.676435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hksxg\" (UniqueName: \"kubernetes.io/projected/52a32494-2804-4187-9e84-942da2f02945-kube-api-access-hksxg\") pod \"test-curl-7629-139657504406960\" (UID: \"52a32494-2804-4187-9e84-942da2f02945\") " pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:15.789609 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.789519 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:15.923797 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:15.923772 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:15.925294 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:13:15.925269 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52a32494_2804_4187_9e84_942da2f02945.slice/crio-51fa3df91744a9ef98e88373000725cde6c9e809a5d43c097f27d8467b6fb2d1 WatchSource:0}: Error finding container 51fa3df91744a9ef98e88373000725cde6c9e809a5d43c097f27d8467b6fb2d1: Status 404 returned error can't find the container with id 51fa3df91744a9ef98e88373000725cde6c9e809a5d43c097f27d8467b6fb2d1 Jul 22 05:13:16.188541 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:16.185876 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4ed64ca2f6aac02f26d9b06a2496ac86953753326c65a417a0c681f5cf7e8fb" Jul 22 05:13:16.188541 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:16.185998 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:16.188541 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:16.187422 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504406960" event={"ID":"52a32494-2804-4187-9e84-942da2f02945","Type":"ContainerStarted","Data":"51fa3df91744a9ef98e88373000725cde6c9e809a5d43c097f27d8467b6fb2d1"} Jul 22 05:13:16.198127 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:13:16.198091 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7629-139657504404512\" is forbidden: User \"system:node:ip-10-0-142-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-182.ec2.internal' and this object" podUID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:16.929990 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:13:16.929953 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7629-139657504404512\" is forbidden: User \"system:node:ip-10-0-142-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-182.ec2.internal' and this object" podUID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" pod="opendatahub/test-curl-7629-139657504404512" Jul 22 05:13:16.931520 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:16.931499 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd" path="/var/lib/kubelet/pods/77f4693a-8cc8-4ab9-a1c6-5bc6fb3f43fd/volumes" Jul 22 05:13:17.195031 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:17.194936 2566 generic.go:356] "Generic (PLEG): container finished" podID="52a32494-2804-4187-9e84-942da2f02945" containerID="6dd5e984863316269ac01fc8dce54094266b48e001e718663cb58adb3d1bcb93" exitCode=0 Jul 22 05:13:17.195031 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:17.195010 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504406960" event={"ID":"52a32494-2804-4187-9e84-942da2f02945","Type":"ContainerDied","Data":"6dd5e984863316269ac01fc8dce54094266b48e001e718663cb58adb3d1bcb93"} Jul 22 05:13:18.348301 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.348274 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:18.370700 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.370667 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7629-139657504406960_52a32494-2804-4187-9e84-942da2f02945/test-curl-7629-139657504406960/0.log" Jul 22 05:13:18.395293 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.395262 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:18.395425 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.395272 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hksxg\" (UniqueName: \"kubernetes.io/projected/52a32494-2804-4187-9e84-942da2f02945-kube-api-access-hksxg\") pod \"52a32494-2804-4187-9e84-942da2f02945\" (UID: \"52a32494-2804-4187-9e84-942da2f02945\") " Jul 22 05:13:18.397454 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.397423 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52a32494-2804-4187-9e84-942da2f02945-kube-api-access-hksxg" (OuterVolumeSpecName: "kube-api-access-hksxg") pod "52a32494-2804-4187-9e84-942da2f02945" (UID: "52a32494-2804-4187-9e84-942da2f02945"). InnerVolumeSpecName "kube-api-access-hksxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:13:18.398420 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.398399 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:18.496857 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.496758 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hksxg\" (UniqueName: \"kubernetes.io/projected/52a32494-2804-4187-9e84-942da2f02945-kube-api-access-hksxg\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:13:18.524924 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.524891 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:18.525344 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.525329 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52a32494-2804-4187-9e84-942da2f02945" containerName="test-curl-7629-139657504406960" Jul 22 05:13:18.525413 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.525347 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="52a32494-2804-4187-9e84-942da2f02945" containerName="test-curl-7629-139657504406960" Jul 22 05:13:18.525467 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.525430 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="52a32494-2804-4187-9e84-942da2f02945" containerName="test-curl-7629-139657504406960" Jul 22 05:13:18.528588 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.528568 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:18.535922 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.535890 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:18.597386 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.597354 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9rqj\" (UniqueName: \"kubernetes.io/projected/e73c1491-b516-4d4b-9d9a-86a628fd5c00-kube-api-access-f9rqj\") pod \"test-curl-7629-139657504406960\" (UID: \"e73c1491-b516-4d4b-9d9a-86a628fd5c00\") " pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:18.698206 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.698158 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9rqj\" (UniqueName: \"kubernetes.io/projected/e73c1491-b516-4d4b-9d9a-86a628fd5c00-kube-api-access-f9rqj\") pod \"test-curl-7629-139657504406960\" (UID: \"e73c1491-b516-4d4b-9d9a-86a628fd5c00\") " pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:18.706270 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.706239 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9rqj\" (UniqueName: \"kubernetes.io/projected/e73c1491-b516-4d4b-9d9a-86a628fd5c00-kube-api-access-f9rqj\") pod \"test-curl-7629-139657504406960\" (UID: \"e73c1491-b516-4d4b-9d9a-86a628fd5c00\") " pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:18.840645 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.840550 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:18.933558 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.933523 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52a32494-2804-4187-9e84-942da2f02945" path="/var/lib/kubelet/pods/52a32494-2804-4187-9e84-942da2f02945/volumes" Jul 22 05:13:18.977051 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:18.977027 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:18.978666 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:13:18.978643 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode73c1491_b516_4d4b_9d9a_86a628fd5c00.slice/crio-3152ca78faf2823b48e91bd7d5881aec2eb5872bea55c62d1ee8e3c902361cf4 WatchSource:0}: Error finding container 3152ca78faf2823b48e91bd7d5881aec2eb5872bea55c62d1ee8e3c902361cf4: Status 404 returned error can't find the container with id 3152ca78faf2823b48e91bd7d5881aec2eb5872bea55c62d1ee8e3c902361cf4 Jul 22 05:13:19.206307 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:19.206276 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504406960" event={"ID":"e73c1491-b516-4d4b-9d9a-86a628fd5c00","Type":"ContainerStarted","Data":"3152ca78faf2823b48e91bd7d5881aec2eb5872bea55c62d1ee8e3c902361cf4"} Jul 22 05:13:19.207576 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:19.207558 2566 scope.go:117] "RemoveContainer" containerID="6dd5e984863316269ac01fc8dce54094266b48e001e718663cb58adb3d1bcb93" Jul 22 05:13:19.207657 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:19.207566 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:20.216522 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:20.216481 2566 generic.go:356] "Generic (PLEG): container finished" podID="e73c1491-b516-4d4b-9d9a-86a628fd5c00" containerID="3c2b88fdcb848727d8f44d4249c03acc3c892c2451289c75c43451bf196f871a" exitCode=0 Jul 22 05:13:20.217059 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:20.216518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7629-139657504406960" event={"ID":"e73c1491-b516-4d4b-9d9a-86a628fd5c00","Type":"ContainerDied","Data":"3c2b88fdcb848727d8f44d4249c03acc3c892c2451289c75c43451bf196f871a"} Jul 22 05:13:21.354286 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.354260 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:21.375659 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.375624 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7629-139657504406960_e73c1491-b516-4d4b-9d9a-86a628fd5c00/test-curl-7629-139657504406960/0.log" Jul 22 05:13:21.403284 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.403250 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:21.407261 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.407233 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7629-139657504406960"] Jul 22 05:13:21.431730 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.431692 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9rqj\" (UniqueName: \"kubernetes.io/projected/e73c1491-b516-4d4b-9d9a-86a628fd5c00-kube-api-access-f9rqj\") pod \"e73c1491-b516-4d4b-9d9a-86a628fd5c00\" (UID: \"e73c1491-b516-4d4b-9d9a-86a628fd5c00\") " Jul 22 05:13:21.433866 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.433843 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73c1491-b516-4d4b-9d9a-86a628fd5c00-kube-api-access-f9rqj" (OuterVolumeSpecName: "kube-api-access-f9rqj") pod "e73c1491-b516-4d4b-9d9a-86a628fd5c00" (UID: "e73c1491-b516-4d4b-9d9a-86a628fd5c00"). InnerVolumeSpecName "kube-api-access-f9rqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 05:13:21.532980 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:21.532898 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9rqj\" (UniqueName: \"kubernetes.io/projected/e73c1491-b516-4d4b-9d9a-86a628fd5c00-kube-api-access-f9rqj\") on node \"ip-10-0-142-182.ec2.internal\" DevicePath \"\"" Jul 22 05:13:22.229700 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:22.229672 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3152ca78faf2823b48e91bd7d5881aec2eb5872bea55c62d1ee8e3c902361cf4" Jul 22 05:13:22.229700 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:22.229677 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:22.240743 ip-10-0-142-182 kubenswrapper[2566]: E0722 05:13:22.240714 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7629-139657504406960\" is forbidden: User \"system:node:ip-10-0-142-182.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-182.ec2.internal' and this object" podUID="e73c1491-b516-4d4b-9d9a-86a628fd5c00" pod="opendatahub/test-curl-7629-139657504406960" Jul 22 05:13:22.931646 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:13:22.931616 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e73c1491-b516-4d4b-9d9a-86a628fd5c00" path="/var/lib/kubelet/pods/e73c1491-b516-4d4b-9d9a-86a628fd5c00/volumes" Jul 22 05:14:05.826465 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:05.826379 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-665b5547bd-7krsg_26e70862-18e5-4a6a-9cd3-89da4a8711f9/authorino/0.log" Jul 22 05:14:11.892244 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:11.892215 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5676cbf8df-t9j2s_86342374-e0bc-46ce-8b6c-6563caab986c/copy-manifests/0.log" Jul 22 05:14:11.898559 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:11.898537 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5676cbf8df-t9j2s_86342374-e0bc-46ce-8b6c-6563caab986c/manager/0.log" Jul 22 05:14:12.250634 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:12.250543 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-96979fff8-m7j7d_2311a276-c41e-4079-b677-bc3e45864ebc/maas-api/0.log" Jul 22 05:14:12.353722 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:12.353691 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744925-94dh5_42272ed7-224e-4600-b117-14a88c8912cc/cleanup/0.log" Jul 22 05:14:13.299232 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.299204 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q_21d3efef-0a3e-43d9-958a-7b0545e4c5a9/util/0.log" Jul 22 05:14:13.306140 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.306118 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q_21d3efef-0a3e-43d9-958a-7b0545e4c5a9/pull/0.log" Jul 22 05:14:13.312661 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.312642 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q_21d3efef-0a3e-43d9-958a-7b0545e4c5a9/extract/0.log" Jul 22 05:14:13.425720 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.425694 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8_484cc7ac-b16a-4f06-b551-bbb7039f28e0/util/0.log" Jul 22 05:14:13.434352 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.434329 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8_484cc7ac-b16a-4f06-b551-bbb7039f28e0/pull/0.log" Jul 22 05:14:13.440484 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.440467 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8_484cc7ac-b16a-4f06-b551-bbb7039f28e0/extract/0.log" Jul 22 05:14:13.557604 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.557523 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx_960943a7-a739-4df1-a4ed-f92a13c4e1d3/util/0.log" Jul 22 05:14:13.564678 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.564638 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx_960943a7-a739-4df1-a4ed-f92a13c4e1d3/pull/0.log" Jul 22 05:14:13.571598 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.571577 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx_960943a7-a739-4df1-a4ed-f92a13c4e1d3/extract/0.log" Jul 22 05:14:13.692266 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.692228 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj_a23533ca-695d-4ae4-a997-283dae661255/util/0.log" Jul 22 05:14:13.699293 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.699272 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj_a23533ca-695d-4ae4-a997-283dae661255/pull/0.log" Jul 22 05:14:13.705764 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.705747 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj_a23533ca-695d-4ae4-a997-283dae661255/extract/0.log" Jul 22 05:14:13.829725 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:13.829639 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-665b5547bd-7krsg_26e70862-18e5-4a6a-9cd3-89da4a8711f9/authorino/0.log" Jul 22 05:14:14.163991 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:14.163964 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-lf7nf_9036e68c-5f88-4940-85e4-8eac981087fd/kuadrant-console-plugin/0.log" Jul 22 05:14:15.277848 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:15.277814 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86559b66b8-w77r2_f12fd811-9472-430a-bd16-02d18b44aae5/kube-auth-proxy/0.log" Jul 22 05:14:15.616248 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:15.616167 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l9gfw_f18d6e2c-20cd-4219-be78-23381e89f5d5/payload-processing/0.log" Jul 22 05:14:16.036097 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:16.036054 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-ffhwm_33e9e8d9-f408-4a63-adee-8105161e533d/main/0.log" Jul 22 05:14:16.353319 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:16.353245 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjfvm5_3922e9e6-49c5-4b5c-a25e-fd673e714e0e/main/0.log" Jul 22 05:14:16.571673 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:16.571647 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-gksb5_36ef2bf8-c50e-4302-9e93-f130e649522f/main/0.log" Jul 22 05:14:25.217261 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:25.217229 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jvssr_a864c73c-bf76-41a9-a509-85976d29df5e/global-pull-secret-syncer/0.log" Jul 22 05:14:25.414218 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:25.414175 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-s6bkv_70691c50-ec06-4b34-966d-0730f0ebca13/konnectivity-agent/0.log" Jul 22 05:14:25.489275 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:25.489191 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-182.ec2.internal_c19c00cd70e4577ed80789f2afae047c/haproxy/0.log" Jul 22 05:14:29.671142 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.671096 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q_21d3efef-0a3e-43d9-958a-7b0545e4c5a9/extract/0.log" Jul 22 05:14:29.704198 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.704169 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q_21d3efef-0a3e-43d9-958a-7b0545e4c5a9/util/0.log" Jul 22 05:14:29.730309 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.730268 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mfc8q_21d3efef-0a3e-43d9-958a-7b0545e4c5a9/pull/0.log" Jul 22 05:14:29.778882 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.778840 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8_484cc7ac-b16a-4f06-b551-bbb7039f28e0/extract/0.log" Jul 22 05:14:29.812311 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.812282 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8_484cc7ac-b16a-4f06-b551-bbb7039f28e0/util/0.log" Jul 22 05:14:29.835518 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.835488 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0j48d8_484cc7ac-b16a-4f06-b551-bbb7039f28e0/pull/0.log" Jul 22 05:14:29.862972 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.862944 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx_960943a7-a739-4df1-a4ed-f92a13c4e1d3/extract/0.log" Jul 22 05:14:29.888981 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.888948 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx_960943a7-a739-4df1-a4ed-f92a13c4e1d3/util/0.log" Jul 22 05:14:29.909741 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.909714 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73vldcx_960943a7-a739-4df1-a4ed-f92a13c4e1d3/pull/0.log" Jul 22 05:14:29.936661 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.936580 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj_a23533ca-695d-4ae4-a997-283dae661255/extract/0.log" Jul 22 05:14:29.963035 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.963007 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj_a23533ca-695d-4ae4-a997-283dae661255/util/0.log" Jul 22 05:14:29.983451 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:29.983426 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1gb5zj_a23533ca-695d-4ae4-a997-283dae661255/pull/0.log" Jul 22 05:14:30.328743 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:30.328643 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-665b5547bd-7krsg_26e70862-18e5-4a6a-9cd3-89da4a8711f9/authorino/0.log" Jul 22 05:14:30.504014 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:30.503982 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-lf7nf_9036e68c-5f88-4940-85e4-8eac981087fd/kuadrant-console-plugin/0.log" Jul 22 05:14:32.809317 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:32.809282 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-46mcd_03e46822-7796-468c-91b6-e72697f0e4ea/cluster-monitoring-operator/0.log" Jul 22 05:14:32.844819 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:32.844789 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-z7ln7_26688b8c-fde4-486e-8527-cf9795f8355b/kube-state-metrics/0.log" Jul 22 05:14:32.870196 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:32.870158 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-z7ln7_26688b8c-fde4-486e-8527-cf9795f8355b/kube-rbac-proxy-main/0.log" Jul 22 05:14:32.891548 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:32.891518 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-z7ln7_26688b8c-fde4-486e-8527-cf9795f8355b/kube-rbac-proxy-self/0.log" Jul 22 05:14:32.916775 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:32.916744 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-56668596dd-4xm4j_0e64a6d2-f1e8-4953-93a5-1beb6732d482/metrics-server/0.log" Jul 22 05:14:32.945759 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:32.945718 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-xnv87_3e838cdb-e77a-485f-b81c-093068166367/monitoring-plugin/0.log" Jul 22 05:14:33.058943 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.058912 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7m876_65a58f4e-fd6d-45c0-8d15-dcbe42d877d9/node-exporter/0.log" Jul 22 05:14:33.079325 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.079242 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7m876_65a58f4e-fd6d-45c0-8d15-dcbe42d877d9/kube-rbac-proxy/0.log" Jul 22 05:14:33.100744 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.100717 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7m876_65a58f4e-fd6d-45c0-8d15-dcbe42d877d9/init-textfile/0.log" Jul 22 05:14:33.204750 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.204722 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-qlfrg_65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931/kube-rbac-proxy-main/0.log" Jul 22 05:14:33.225390 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.225361 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-qlfrg_65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931/kube-rbac-proxy-self/0.log" Jul 22 05:14:33.251417 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.251378 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-qlfrg_65e9d171-e5f7-4f44-a2e7-5f2bcc1c4931/openshift-state-metrics/0.log" Jul 22 05:14:33.644843 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.644810 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8674fd69d-5t2vd_d7457d92-17c3-4353-9cd9-a88214127be4/thanos-query/0.log" Jul 22 05:14:33.668269 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.668238 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8674fd69d-5t2vd_d7457d92-17c3-4353-9cd9-a88214127be4/kube-rbac-proxy-web/0.log" Jul 22 05:14:33.689332 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.689301 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8674fd69d-5t2vd_d7457d92-17c3-4353-9cd9-a88214127be4/kube-rbac-proxy/0.log" Jul 22 05:14:33.719518 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.719489 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8674fd69d-5t2vd_d7457d92-17c3-4353-9cd9-a88214127be4/prom-label-proxy/0.log" Jul 22 05:14:33.739949 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.739912 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8674fd69d-5t2vd_d7457d92-17c3-4353-9cd9-a88214127be4/kube-rbac-proxy-rules/0.log" Jul 22 05:14:33.762903 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:33.762877 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-8674fd69d-5t2vd_d7457d92-17c3-4353-9cd9-a88214127be4/kube-rbac-proxy-metrics/0.log" Jul 22 05:14:34.890387 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:34.890341 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-4s6vm_63329226-43a6-497b-86cd-bc6736e1edf5/networking-console-plugin/0.log" Jul 22 05:14:35.539305 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.539266 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn"] Jul 22 05:14:35.539828 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.539805 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e73c1491-b516-4d4b-9d9a-86a628fd5c00" containerName="test-curl-7629-139657504406960" Jul 22 05:14:35.539828 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.539827 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73c1491-b516-4d4b-9d9a-86a628fd5c00" containerName="test-curl-7629-139657504406960" Jul 22 05:14:35.539997 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.539959 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e73c1491-b516-4d4b-9d9a-86a628fd5c00" containerName="test-curl-7629-139657504406960" Jul 22 05:14:35.543157 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.543137 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.546204 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.546185 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c7bkb\"/\"kube-root-ca.crt\"" Jul 22 05:14:35.546322 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.546232 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c7bkb\"/\"default-dockercfg-c4nsx\"" Jul 22 05:14:35.546322 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.546232 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c7bkb\"/\"openshift-service-ca.crt\"" Jul 22 05:14:35.552042 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.552021 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn"] Jul 22 05:14:35.708495 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.708453 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-podres\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.708672 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.708503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdv9x\" (UniqueName: \"kubernetes.io/projected/f8379248-dc22-407d-bdaf-c80cb7e78997-kube-api-access-hdv9x\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.708672 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.708544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-sys\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.708672 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.708632 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-lib-modules\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.708782 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.708673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-proc\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809227 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-lib-modules\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809227 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-proc\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809227 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809216 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-podres\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809481 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdv9x\" (UniqueName: \"kubernetes.io/projected/f8379248-dc22-407d-bdaf-c80cb7e78997-kube-api-access-hdv9x\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809481 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809267 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-sys\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809481 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809343 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-lib-modules\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809481 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809367 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-proc\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809481 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809366 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-sys\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.809481 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.809370 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f8379248-dc22-407d-bdaf-c80cb7e78997-podres\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.818118 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.818063 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdv9x\" (UniqueName: \"kubernetes.io/projected/f8379248-dc22-407d-bdaf-c80cb7e78997-kube-api-access-hdv9x\") pod \"perf-node-gather-daemonset-hqfdn\" (UID: \"f8379248-dc22-407d-bdaf-c80cb7e78997\") " pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.853583 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.853553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:35.988439 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:35.988410 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn"] Jul 22 05:14:35.991186 ip-10-0-142-182 kubenswrapper[2566]: W0722 05:14:35.991161 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf8379248_dc22_407d_bdaf_c80cb7e78997.slice/crio-edf059a1f0b942f0b556982aa212cfbce535b40be9dfe828f391f7f7df78b58a WatchSource:0}: Error finding container edf059a1f0b942f0b556982aa212cfbce535b40be9dfe828f391f7f7df78b58a: Status 404 returned error can't find the container with id edf059a1f0b942f0b556982aa212cfbce535b40be9dfe828f391f7f7df78b58a Jul 22 05:14:36.166715 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.166683 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-58578f969f-ndgrz_68e16db6-39ea-4c19-8125-f69b9a315485/console/0.log" Jul 22 05:14:36.200729 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.200684 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-nkzrk_ba7434a6-9f90-47d6-9451-261245769db1/download-server/0.log" Jul 22 05:14:36.664171 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.664129 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" event={"ID":"f8379248-dc22-407d-bdaf-c80cb7e78997","Type":"ContainerStarted","Data":"89834b3c79faadbfdfcd079abc72241aee9f52d6d264c9983ecda7b0a44f7c20"} Jul 22 05:14:36.664375 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.664231 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" event={"ID":"f8379248-dc22-407d-bdaf-c80cb7e78997","Type":"ContainerStarted","Data":"edf059a1f0b942f0b556982aa212cfbce535b40be9dfe828f391f7f7df78b58a"} Jul 22 05:14:36.664375 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.664266 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:36.682117 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.682048 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" podStartSLOduration=1.682031225 podStartE2EDuration="1.682031225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 05:14:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 05:14:36.679489848 +0000 UTC m=+3000.768769039" watchObservedRunningTime="2026-07-22 05:14:36.682031225 +0000 UTC m=+3000.771310396" Jul 22 05:14:36.800818 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:36.800783 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-vwnmd_a9f079ae-df39-4aa7-bb6c-222749daaab1/volume-data-source-validator/0.log" Jul 22 05:14:37.772922 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:37.772896 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4c7tk_c1d161be-db7d-4a6b-a1e1-7b54fdb04260/dns/0.log" Jul 22 05:14:37.794754 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:37.794730 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4c7tk_c1d161be-db7d-4a6b-a1e1-7b54fdb04260/kube-rbac-proxy/0.log" Jul 22 05:14:37.942696 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:37.942671 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-96kj6_3a848e11-271b-4c2f-bdb3-4d4ac66fc3c3/dns-node-resolver/0.log" Jul 22 05:14:38.572492 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:38.572456 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7vpdn_bb43c786-c2c2-459e-bb69-7a661f7216ae/node-ca/0.log" Jul 22 05:14:39.740952 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:39.740926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86559b66b8-w77r2_f12fd811-9472-430a-bd16-02d18b44aae5/kube-auth-proxy/0.log" Jul 22 05:14:39.881745 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:39.881714 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l9gfw_f18d6e2c-20cd-4219-be78-23381e89f5d5/payload-processing/0.log" Jul 22 05:14:40.597705 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:40.597673 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hdzxn_a1d48f3e-a7cf-4d48-9966-445390f29897/serve-healthcheck-canary/0.log" Jul 22 05:14:41.237775 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:41.237743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnrwz_4c014aef-a873-4c15-8033-04ccd2990813/kube-rbac-proxy/0.log" Jul 22 05:14:41.257634 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:41.257605 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnrwz_4c014aef-a873-4c15-8033-04ccd2990813/exporter/0.log" Jul 22 05:14:41.280047 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:41.280018 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tnrwz_4c014aef-a873-4c15-8033-04ccd2990813/extractor/0.log" Jul 22 05:14:42.683399 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:42.683372 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c7bkb/perf-node-gather-daemonset-hqfdn" Jul 22 05:14:44.072590 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:44.072553 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5676cbf8df-t9j2s_86342374-e0bc-46ce-8b6c-6563caab986c/manager/0.log" Jul 22 05:14:44.093805 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:44.093778 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5676cbf8df-t9j2s_86342374-e0bc-46ce-8b6c-6563caab986c/copy-manifests/0.log" Jul 22 05:14:45.562099 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:45.561995 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-596745447d-sljz5_9714745a-4e1e-4bb3-afdf-8d9d84d09e0b/manager/0.log" Jul 22 05:14:45.586496 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:45.586473 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wbk7n_e6f83e1f-b407-4b86-8d8d-e1a3e93afedc/openshift-lws-operator/0.log" Jul 22 05:14:46.209795 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:46.209763 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86559b66b8-w77r2_f12fd811-9472-430a-bd16-02d18b44aae5/kube-auth-proxy/0.log" Jul 22 05:14:46.321954 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:46.321923 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l9gfw_f18d6e2c-20cd-4219-be78-23381e89f5d5/payload-processing/0.log" Jul 22 05:14:47.782013 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:47.781986 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86559b66b8-w77r2_f12fd811-9472-430a-bd16-02d18b44aae5/kube-auth-proxy/0.log" Jul 22 05:14:47.903959 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:47.903926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l9gfw_f18d6e2c-20cd-4219-be78-23381e89f5d5/payload-processing/0.log" Jul 22 05:14:50.659731 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:50.659697 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ppxvh_6d218c1c-8cb0-4dee-a4f9-db19ce62723c/migrator/0.log" Jul 22 05:14:50.678504 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:50.678477 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-ppxvh_6d218c1c-8cb0-4dee-a4f9-db19ce62723c/graceful-termination/0.log" Jul 22 05:14:52.139166 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.139136 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/kube-multus-additional-cni-plugins/0.log" Jul 22 05:14:52.159678 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.159645 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/egress-router-binary-copy/0.log" Jul 22 05:14:52.182232 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.182210 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/cni-plugins/0.log" Jul 22 05:14:52.203874 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.203845 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/bond-cni-plugin/0.log" Jul 22 05:14:52.236674 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.236649 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/routeoverride-cni/0.log" Jul 22 05:14:52.257673 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.257647 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/whereabouts-cni-bincopy/0.log" Jul 22 05:14:52.278707 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.278681 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-crnp4_c091b74a-8aef-4cb2-8042-67b4e153cdb7/whereabouts-cni/0.log" Jul 22 05:14:52.645964 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.645935 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z9c8z_eeed6b91-c5c1-4555-b6a4-9d2bd34fe7ef/kube-multus/0.log" Jul 22 05:14:52.712870 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.712843 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4fnp2_256b8fe7-db10-4e43-9d5c-d75acc517fb2/network-metrics-daemon/0.log" Jul 22 05:14:52.730461 ip-10-0-142-182 kubenswrapper[2566]: I0722 05:14:52.730433 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4fnp2_256b8fe7-db10-4e43-9d5c-d75acc517fb2/kube-rbac-proxy/0.log"