Jul 13 18:58:14.556298 ip-10-0-133-70 systemd[1]: Starting Kubernetes Kubelet... Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.001230 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005875 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005894 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005898 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005901 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005905 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005908 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005911 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:15.109998 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005913 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:15.095418 ip-10-0-133-70 systemd[1]: Started Kubernetes Kubelet. Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005916 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005919 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005923 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005928 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005931 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005934 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005937 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005940 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005943 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005945 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005948 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005967 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005973 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005977 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005980 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005983 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005986 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005989 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:15.123562 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005992 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005995 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.005998 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006001 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006003 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006006 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006009 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006011 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006014 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006016 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006019 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006022 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006026 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006028 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006031 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006034 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006036 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006040 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006043 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006046 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:15.131299 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006049 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006051 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006054 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006056 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006060 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006062 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006065 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006067 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006069 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006072 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006075 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006077 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006080 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006082 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006084 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006087 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006089 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006092 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006094 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006097 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:15.133799 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006099 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006102 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006104 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006107 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006109 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006112 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006114 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006117 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006119 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006123 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006126 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006129 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006131 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006134 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006137 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006148 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006154 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006160 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006163 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:15.135671 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006166 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006169 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006172 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006175 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006177 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006180 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006182 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006185 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006188 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006191 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006193 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006196 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006198 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006201 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006926 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006933 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006936 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006939 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006942 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006944 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:15.138922 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006947 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006950 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006976 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006979 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006982 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006985 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006987 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006990 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006995 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.006998 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007002 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007006 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007008 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007011 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007014 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007017 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007020 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007022 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007025 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:15.151812 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007027 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007030 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007032 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007035 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007038 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007040 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007043 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007045 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007048 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007051 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007053 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007056 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007058 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007062 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007065 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007067 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007070 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007072 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007075 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007077 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:15.291077 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007080 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007083 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007085 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007088 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007090 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007093 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007096 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007098 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007101 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007104 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007106 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007110 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007113 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007116 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007119 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007124 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007127 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007130 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007133 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:15.301417 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007136 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007139 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007141 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007144 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007146 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007149 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007152 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007155 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007158 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007161 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007163 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007166 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007169 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007171 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007174 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007177 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007179 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007182 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007184 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007190 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:15.302677 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007192 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007195 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007199 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007203 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007206 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007209 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007212 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007215 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007217 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007220 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007223 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007225 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007228 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007230 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007233 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007628 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007633 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007635 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007638 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007641 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:15.304206 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007644 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007647 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007650 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007653 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007655 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007658 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007661 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007664 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007666 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007669 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007672 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007677 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007680 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007683 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007686 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007689 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007692 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007695 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007697 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:15.305337 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007700 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007703 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007705 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007708 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007711 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007713 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007717 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007719 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007722 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007726 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007729 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007733 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007736 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007738 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007741 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007744 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007746 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007749 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007752 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007754 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:15.305909 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007760 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007763 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007766 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007769 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007771 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007775 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007778 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007780 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007783 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007786 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007788 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007791 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007794 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007797 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007799 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007802 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007804 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007807 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007810 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007813 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:15.306523 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007815 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007818 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007820 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007823 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007825 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007828 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007830 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007833 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007837 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007840 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007844 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007847 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007852 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007855 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007858 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007861 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007865 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007868 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007871 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007874 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:15.307082 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007877 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007879 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007882 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007885 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007888 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007890 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007893 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007896 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007898 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007901 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007903 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007906 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007908 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007911 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.007913 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008039 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008076 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008086 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008091 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008095 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008099 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 18:58:15.307703 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008104 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008108 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008111 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008114 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008118 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008122 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008125 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008128 2580 flags.go:64] FLAG: --cgroup-root="" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008131 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008134 2580 flags.go:64] FLAG: --client-ca-file="" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008138 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008141 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008146 2580 flags.go:64] FLAG: --cluster-domain="" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008149 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008152 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008155 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008159 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008163 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008166 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008172 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008175 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008179 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008182 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008185 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 18:58:15.308297 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008188 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008192 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008197 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008200 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008203 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008206 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008210 2580 flags.go:64] FLAG: --enable-server="true" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008213 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008220 2580 flags.go:64] FLAG: --event-burst="100" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008224 2580 flags.go:64] FLAG: --event-qps="50" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008227 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008230 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008233 2580 flags.go:64] FLAG: --eviction-hard="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008237 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008240 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008243 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008246 2580 flags.go:64] FLAG: --eviction-soft="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008249 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008252 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008255 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008258 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008261 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008264 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008267 2580 flags.go:64] FLAG: --feature-gates="" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008272 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 18:58:15.309044 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008275 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008278 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008283 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008286 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008289 2580 flags.go:64] FLAG: --help="false" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008292 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-133-70.ec2.internal" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008295 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008298 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008301 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008306 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008310 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008313 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008316 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008318 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008321 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008324 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008327 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008330 2580 flags.go:64] FLAG: --kube-reserved="" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008333 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008336 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008338 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008341 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008344 2580 flags.go:64] FLAG: --lock-file="" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008347 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 18:58:15.309720 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008350 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008353 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008358 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008362 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008365 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008368 2580 flags.go:64] FLAG: --logging-format="text" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008371 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008374 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008377 2580 flags.go:64] FLAG: --manifest-url="" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008380 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008387 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008390 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008394 2580 flags.go:64] FLAG: --max-pods="110" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008397 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008400 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008403 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008406 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008410 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008413 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008416 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008425 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008428 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008431 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 18:58:15.310751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008434 2580 flags.go:64] FLAG: --pod-cidr="" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008437 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008444 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008447 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008450 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008453 2580 flags.go:64] FLAG: --port="10250" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008456 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008459 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f5d0bdeff6d6550b" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008462 2580 flags.go:64] FLAG: --qos-reserved="" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008465 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008468 2580 flags.go:64] FLAG: --register-node="true" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008471 2580 flags.go:64] FLAG: --register-with-taints="" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008475 2580 flags.go:64] FLAG: --registry-burst="10" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008478 2580 flags.go:64] FLAG: --registry-qps="5" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008481 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008484 2580 flags.go:64] FLAG: --reserved-memory="" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008488 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008491 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008494 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008497 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008502 2580 flags.go:64] FLAG: --runonce="false" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008505 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008508 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008511 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008514 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 18:58:15.311888 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008517 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008524 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008527 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008530 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008533 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008536 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008539 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008542 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008546 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008549 2580 flags.go:64] FLAG: --system-cgroups="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008552 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008558 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008561 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008564 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008568 2580 flags.go:64] FLAG: --tls-min-version="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008572 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008575 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008577 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008580 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008583 2580 flags.go:64] FLAG: --v="2" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008587 2580 flags.go:64] FLAG: --version="false" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008592 2580 flags.go:64] FLAG: --vmodule="" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008596 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.008599 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008724 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:15.312668 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008729 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008732 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008737 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008740 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008743 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008747 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008750 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008753 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008758 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008761 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008764 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008767 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008770 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008772 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008775 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008778 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008781 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008783 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008788 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008792 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:15.502207 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008794 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008797 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008800 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008803 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008806 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008808 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008811 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008814 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008816 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008819 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008822 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008825 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008827 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008830 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008834 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008837 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008839 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008842 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008845 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008848 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:15.502914 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008852 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008855 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008858 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008860 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008863 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008867 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008869 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008872 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008875 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008877 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008880 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008883 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008886 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008888 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008891 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008893 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008896 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008898 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008901 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:15.503539 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008904 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008906 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008909 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008913 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008916 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008920 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008923 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008927 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008930 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008933 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008935 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008938 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008941 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008945 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008948 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008951 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008970 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008973 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008976 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008979 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:15.504215 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008982 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008986 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008989 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008992 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008995 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.008998 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009000 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009003 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009006 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009008 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009011 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009014 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009017 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009019 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009022 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009025 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009028 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009031 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:15.504781 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.009033 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.009870 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.020143 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.020167 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.021056 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.021068 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021140 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021148 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021152 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021156 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021159 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021162 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021166 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021168 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021171 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:15.505323 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021173 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021176 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021179 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021181 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021184 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021187 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021210 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021214 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021218 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021221 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021224 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021227 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021237 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021241 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021244 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021247 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021249 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021252 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021255 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021259 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:15.505748 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021271 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021275 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021280 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021283 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021287 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021289 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021292 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021295 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021297 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021300 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021303 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021306 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021308 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021311 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021313 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021316 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021319 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021321 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021324 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021327 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:15.506467 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021330 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021332 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021335 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021337 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021340 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021343 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021345 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021348 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021351 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021353 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021356 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021362 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021365 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021380 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021383 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021386 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021390 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021392 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021395 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021398 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:15.507357 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021400 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021403 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021406 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021409 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021411 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021414 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021417 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021419 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021424 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021427 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021430 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021433 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021436 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021439 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021441 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021444 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021447 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021450 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021452 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:15.508270 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021455 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021457 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021460 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021462 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021466 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021469 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021472 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021484 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021487 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021492 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021496 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.021502 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021661 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021667 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 18:58:15.509163 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021670 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021674 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021676 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021679 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021682 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021684 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021687 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021690 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021692 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021695 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021697 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021700 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021703 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021705 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021708 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021710 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021713 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021715 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021718 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021721 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 18:58:15.510233 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021723 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021726 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021728 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021732 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021735 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021737 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021746 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021750 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021752 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021755 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021758 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021760 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021764 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021768 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021771 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021774 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021777 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021780 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021783 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 18:58:15.511084 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021785 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021788 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021790 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021793 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021796 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021798 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021801 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021804 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021806 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021809 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021811 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021814 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021816 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021819 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021821 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021824 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021828 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021831 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021833 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021836 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 18:58:15.511975 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021848 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021851 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021854 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021856 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021859 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021862 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021864 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021867 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021869 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021872 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021874 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021878 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021880 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021883 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021885 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021888 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021891 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021895 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 18:58:15.512844 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021897 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021901 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021905 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021908 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021911 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021914 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021917 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021920 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021922 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021925 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021929 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021932 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021935 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021938 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021941 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021950 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021971 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021974 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021976 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 18:58:15.513919 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:15.021979 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.021985 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.022134 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.024750 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.026653 2580 server.go:1019] "Starting client certificate rotation" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.026765 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.026804 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.029529 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.036500 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.050013 2580 log.go:25] "Validated CRI v1 runtime API" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.054194 2580 log.go:25] "Validated CRI v1 image API" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.056495 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 18:58:15.514807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.059195 2580 fs.go:135] Filesystem UUIDs: map[653a0888-6b48-4e4e-b4ee-79366a26685a:/dev/nvme0n1p4 79af82b3-713f-4d7c-a291-be129be734db:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 13 18:58:15.515464 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.059214 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 18:58:15.515464 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.062000 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.064501 2580 manager.go:216] Machine: {Timestamp:2026-07-13 18:58:15.06307897 +0000 UTC m=+0.377352422 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3072090 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2038dbbb3f0481abdf7e959746ee16 SystemUUID:ec2038db-bb3f-0481-abdf-7e959746ee16 BootID:6387350b-3b44-457d-bf1f-f89de0de8471 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4c:a5:ba:b8:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4c:a5:ba:b8:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c6:52:70:6a:39:99 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.064619 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.064712 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.068594 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.068636 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-70.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.069458 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.069469 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.069478 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.070319 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.072306 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076390 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076544 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076574 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076590 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076606 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076616 2580 kubelet.go:398] "Adding apiserver pod source" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.076625 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 18:58:15.515890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.077158 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.077174 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.079600 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.082150 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.082189 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083044 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083071 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083077 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083083 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083092 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083098 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083108 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083114 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083130 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083136 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083144 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.083150 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.085063 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.085071 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.093906 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.093906 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.093982 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-70.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.094503 2580 server.go:1268] "Started kubelet" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.094589 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.094664 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.094726 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.096229 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.096732 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.099027 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.099241 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-70.ec2.internal.18c1ee70ab614679 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-70.ec2.internal,UID:ip-10-0-133-70.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-70.ec2.internal,},FirstTimestamp:2026-07-13 18:58:15.094453881 +0000 UTC m=+0.408727577,LastTimestamp:2026-07-13 18:58:15.094453881 +0000 UTC m=+0.408727577,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-70.ec2.internal,}" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.104760 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.104768 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.104907 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105473 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105479 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105524 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105624 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105635 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105642 2580 factory.go:55] Registering systemd factory Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105669 2580 factory.go:223] Registration of the systemd container factory successfully Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105900 2580 factory.go:153] Registering CRI-O factory Jul 13 18:58:15.518860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105915 2580 factory.go:223] Registration of the crio container factory successfully Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.105997 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.106062 2580 factory.go:103] Registering Raw factory Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.106077 2580 manager.go:1196] Started watching for new ooms in manager Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.106990 2580 manager.go:349] Starting recovery of all containers Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.116397 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cfnxs" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.118186 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.118170 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-70.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.118613 2580 manager.go:354] Recovery completed Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.124343 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.124763 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cfnxs" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.126599 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.126636 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.126652 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.127362 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.127374 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.127378 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.127393 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.128114 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.128146 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.128157 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.128510 2580 policy_none.go:49] "None policy: Start" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.128524 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.128536 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.129449 2580 policy_none.go:47] "Start" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.205206 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.253298 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.254542 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.254581 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.254608 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.254649 2580 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.257385 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.306295 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.325376 2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-70.ec2.internal\" not found" node="ip-10-0-133-70.ec2.internal" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.338724 2580 manager.go:337] "Starting Device Plugin manager" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.338816 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.338832 2580 server.go:87] "Starting device plugin registration server" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.339121 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.339132 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.339249 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.339334 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.339342 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.339887 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.339941 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:15.520606 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.355408 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal"] Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.355487 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.356535 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.356563 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.356573 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.357934 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.358076 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.358106 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.361815 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.361841 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.361817 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.361885 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.361851 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.361899 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.363185 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.363212 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.364050 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.364075 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.364086 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.396874 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-70.ec2.internal\" not found" node="ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.401518 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-70.ec2.internal\" not found" node="ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.407189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5251b4cf7c0a51ecb6428df168d891e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal\" (UID: \"5251b4cf7c0a51ecb6428df168d891e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.407214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5251b4cf7c0a51ecb6428df168d891e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal\" (UID: \"5251b4cf7c0a51ecb6428df168d891e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.407242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c05c575dd3a415123dd60cca6ef09a9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-70.ec2.internal\" (UID: \"0c05c575dd3a415123dd60cca6ef09a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.439559 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.440514 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientMemory" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.440542 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.440551 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeHasSufficientPID" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.440583 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.450309 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.450339 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-70.ec2.internal\": node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.481419 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:16.997362 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.507670 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c05c575dd3a415123dd60cca6ef09a9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-70.ec2.internal\" (UID: \"0c05c575dd3a415123dd60cca6ef09a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.507699 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5251b4cf7c0a51ecb6428df168d891e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal\" (UID: \"5251b4cf7c0a51ecb6428df168d891e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.507718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5251b4cf7c0a51ecb6428df168d891e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal\" (UID: \"5251b4cf7c0a51ecb6428df168d891e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.507747 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5251b4cf7c0a51ecb6428df168d891e3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal\" (UID: \"5251b4cf7c0a51ecb6428df168d891e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.507785 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c05c575dd3a415123dd60cca6ef09a9-config\") pod \"kube-apiserver-proxy-ip-10-0-133-70.ec2.internal\" (UID: \"0c05c575dd3a415123dd60cca6ef09a9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.508115 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5251b4cf7c0a51ecb6428df168d891e3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal\" (UID: \"5251b4cf7c0a51ecb6428df168d891e3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.582287 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:15.682990 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.699153 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" Jul 13 18:58:16.999186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:15.704921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:17.178195 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:17.069128 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:17.178195 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:17.169244 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:17.397625 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.233164 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:17.397625 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:17.269553 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:17.397625 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.311147 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 18:58:17.397625 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.311313 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 18:58:17.397625 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.311325 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 18:58:17.397625 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:17.370559 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:17.560335 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.406979 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:17.560335 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.416293 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 18:53:16 +0000 UTC" deadline="2028-03-08 11:12:43.899428012 +0000 UTC" Jul 13 18:58:17.560335 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.416326 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14488h14m26.483105055s" Jul 13 18:58:17.560335 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:17.471454 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-70.ec2.internal\" not found" Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.564920 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:17.567473 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c05c575dd3a415123dd60cca6ef09a9.slice/crio-9fd4522aeb4819dbfade2c79375fa53c061257f5514e1d5882ff603509049e4d WatchSource:0}: Error finding container 9fd4522aeb4819dbfade2c79375fa53c061257f5514e1d5882ff603509049e4d: Status 404 returned error can't find the container with id 9fd4522aeb4819dbfade2c79375fa53c061257f5514e1d5882ff603509049e4d Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.573579 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.591027 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.604554 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.605384 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" Jul 13 18:58:17.745790 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:17.623533 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.363781 2580 apiserver.go:52] "Watching apiserver" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.372642 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.372985 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-bkvh6","openshift-cluster-node-tuning-operator/tuned-gfdbr","openshift-dns/node-resolver-fmnt7","openshift-image-registry/node-ca-pkh94","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal","openshift-multus/multus-additional-cni-plugins-8l58j","openshift-multus/network-metrics-daemon-bpzhc","openshift-network-diagnostics/network-check-target-vk9kv","kube-system/global-pull-secret-syncer-zsgck","kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548","openshift-multus/multus-hmn6n","openshift-network-operator/iptables-alerter-tf9rh","openshift-ovn-kubernetes/ovnkube-node-nmg8g"] Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.374760 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.375738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.376655 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.377180 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.377632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.377681 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.377634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-fs6zk\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.377962 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.378281 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lsnbz\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.378468 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.378611 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.378626 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.379104 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.379457 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fmmcw\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.379528 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.379740 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.380086 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5rpcd\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.380427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.381611 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.381661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.382688 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.382982 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.383898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.385274 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.385332 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.386550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:18.452850 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.386599 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.386651 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.386886 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.386905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.387876 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.387921 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.387928 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.387877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8k96r\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.388019 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.388193 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.388404 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.388401 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.389323 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.389337 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.389351 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.389378 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kgz75\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.389387 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.389449 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.390462 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.390731 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.391942 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-g6n5d\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.392181 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-lk2dz\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.392365 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.392504 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.392567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.392709 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hph7t\"" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.399830 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.401456 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-etc-kubernetes\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gzb6\" (UniqueName: \"kubernetes.io/projected/1022377c-cbde-4f08-8540-b82161f2f16a-kube-api-access-8gzb6\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404567 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-sys-fs\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404594 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-var-lib-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.454371 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-log-socket\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-ovnkube-script-lib\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404694 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-hostroot\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5hsq\" (UniqueName: \"kubernetes.io/projected/c0f5b59c-9501-4223-abfd-30dad4473397-kube-api-access-m5hsq\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-os-release\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctcrb\" (UniqueName: \"kubernetes.io/projected/4061c19f-9533-4feb-bcf2-3f42c037565e-kube-api-access-ctcrb\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404786 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-system-cni-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404807 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-cni-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1022377c-cbde-4f08-8540-b82161f2f16a-cni-binary-copy\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404850 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0f5b59c-9501-4223-abfd-30dad4473397-ovn-node-metrics-cert\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404888 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404925 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-tuned\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysconfig\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.404998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysctl-d\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.456339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-var-lib-kubelet\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-netns\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-conf-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-host-slash\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-ovnkube-config\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405163 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa191c78-a11e-4519-a32b-c036c00c12cf-host\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405185 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-dbus\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405208 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-socket-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405234 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l5v4\" (UniqueName: \"kubernetes.io/projected/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-kube-api-access-7l5v4\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-cnibin\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-tmp\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdcmq\" (UniqueName: \"kubernetes.io/projected/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-kube-api-access-xdcmq\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8fll\" (UniqueName: \"kubernetes.io/projected/06c1ae6a-23e4-4530-8071-496c47f2f077-kube-api-access-w8fll\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-k8s-cni-cncf-io\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405449 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b4404b1-b307-4b37-b215-b250669247b7-tmp-dir\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-kubelet\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.456914 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-ovn\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nnjh\" (UniqueName: \"kubernetes.io/projected/fa191c78-a11e-4519-a32b-c036c00c12cf-kube-api-access-8nnjh\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405543 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-registration-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405619 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-run-netns\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-lib-modules\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysctl-conf\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405751 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b4404b1-b307-4b37-b215-b250669247b7-hosts-file\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405776 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpzm4\" (UniqueName: \"kubernetes.io/projected/0b4404b1-b307-4b37-b215-b250669247b7-kube-api-access-xpzm4\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405800 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-systemd-units\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfxq7\" (UniqueName: \"kubernetes.io/projected/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-kube-api-access-hfxq7\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-systemd\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.405980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-node-log\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-cni-netd\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.457612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406030 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-env-overrides\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406066 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-kubernetes\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406078 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-run\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-cni-multus\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-slash\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406135 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406179 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-device-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-host\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406236 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7-agent-certs\") pod \"konnectivity-agent-bkvh6\" (UID: \"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7\") " pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-os-release\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406282 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1022377c-cbde-4f08-8540-b82161f2f16a-multus-daemon-config\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-cni-bin\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406319 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-modprobe-d\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-systemd\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.458242 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406346 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-cni-bin\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406362 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-cnibin\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406376 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-iptables-alerter-script\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406442 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-socket-dir-parent\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406456 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-kubelet\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-etc-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-system-cni-dir\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa191c78-a11e-4519-a32b-c036c00c12cf-serviceca\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406561 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-kubelet-config\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-sys\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7-konnectivity-ca\") pod \"konnectivity-agent-bkvh6\" (UID: \"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7\") " pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.406648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-multus-certs\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.424851 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kz7sv" Jul 13 18:58:18.458965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.435304 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kz7sv" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507131 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysconfig\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysctl-d\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507184 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-var-lib-kubelet\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507206 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-netns\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507204 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-conf-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507260 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysconfig\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-netns\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507278 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507275 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-var-lib-kubelet\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-host-slash\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507265 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-conf-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507312 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-host-slash\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507317 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-ovnkube-config\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507331 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysctl-d\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.607876 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa191c78-a11e-4519-a32b-c036c00c12cf-host\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-dbus\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-socket-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507476 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fa191c78-a11e-4519-a32b-c036c00c12cf-host\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l5v4\" (UniqueName: \"kubernetes.io/projected/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-kube-api-access-7l5v4\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-cnibin\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507526 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-dbus\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507557 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-cnibin\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507567 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-tmp\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-socket-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507592 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdcmq\" (UniqueName: \"kubernetes.io/projected/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-kube-api-access-xdcmq\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507587 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-run-ovn-kubernetes\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8fll\" (UniqueName: \"kubernetes.io/projected/06c1ae6a-23e4-4530-8071-496c47f2f077-kube-api-access-w8fll\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-k8s-cni-cncf-io\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b4404b1-b307-4b37-b215-b250669247b7-tmp-dir\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-kubelet\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.608633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507749 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-ovn\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507774 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nnjh\" (UniqueName: \"kubernetes.io/projected/fa191c78-a11e-4519-a32b-c036c00c12cf-kube-api-access-8nnjh\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507806 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-registration-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-k8s-cni-cncf-io\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507836 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507864 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-kubelet\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507884 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507906 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-ovn\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507915 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-registration-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507955 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-run-netns\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.507980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-lib-modules\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysctl-conf\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508024 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-run-netns\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b4404b1-b307-4b37-b215-b250669247b7-hosts-file\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b4404b1-b307-4b37-b215-b250669247b7-tmp-dir\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0b4404b1-b307-4b37-b215-b250669247b7-hosts-file\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-lib-modules\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-ovnkube-config\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609196 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508181 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-sysctl-conf\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpzm4\" (UniqueName: \"kubernetes.io/projected/0b4404b1-b307-4b37-b215-b250669247b7-kube-api-access-xpzm4\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-systemd-units\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfxq7\" (UniqueName: \"kubernetes.io/projected/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-kube-api-access-hfxq7\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-systemd\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-node-log\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-cni-netd\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-env-overrides\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508649 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-systemd-units\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508708 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-cni-netd\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508766 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-node-log\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-kubernetes\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-run\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.609972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-cni-multus\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-slash\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-kubernetes\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508947 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-cni-multus\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508957 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-slash\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.508889 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.508984 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-device-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509011 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-run\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-env-overrides\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.509051 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:58:19.009027249 +0000 UTC m=+3.037585591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-device-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-host\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7-agent-certs\") pod \"konnectivity-agent-bkvh6\" (UID: \"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7\") " pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509096 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-host\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-os-release\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.610492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1022377c-cbde-4f08-8540-b82161f2f16a-multus-daemon-config\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-cni-bin\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-run-systemd\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-modprobe-d\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-systemd\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.509310 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509312 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-cni-bin\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-cni-bin\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509366 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.509373 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:19.009339847 +0000 UTC m=+3.037898179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-cnibin\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509418 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-systemd\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509425 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-modprobe-d\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-cni-bin\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-iptables-alerter-script\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-os-release\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509425 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.611294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509472 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-cnibin\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509498 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509523 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-socket-dir-parent\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-kubelet\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509609 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-etc-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509636 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-system-cni-dir\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa191c78-a11e-4519-a32b-c036c00c12cf-serviceca\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509712 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-kubelet-config\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509720 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-var-lib-kubelet\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-sys\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7-konnectivity-ca\") pod \"konnectivity-agent-bkvh6\" (UID: \"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7\") " pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1022377c-cbde-4f08-8540-b82161f2f16a-multus-daemon-config\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-multus-certs\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509801 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-socket-dir-parent\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.611870 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509852 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-kubelet-config\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509870 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-host-run-multus-certs\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509882 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-etc-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509912 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-iptables-alerter-script\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509917 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-system-cni-dir\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-etc-kubernetes\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509977 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-sys\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.509979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gzb6\" (UniqueName: \"kubernetes.io/projected/1022377c-cbde-4f08-8540-b82161f2f16a-kube-api-access-8gzb6\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510015 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-etc-kubernetes\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-sys-fs\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510047 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-var-lib-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-sys-fs\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-log-socket\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-ovnkube-script-lib\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-hostroot\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5hsq\" (UniqueName: \"kubernetes.io/projected/c0f5b59c-9501-4223-abfd-30dad4473397-kube-api-access-m5hsq\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-os-release\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.612572 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctcrb\" (UniqueName: \"kubernetes.io/projected/4061c19f-9533-4feb-bcf2-3f42c037565e-kube-api-access-ctcrb\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-system-cni-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510296 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-log-socket\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510353 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-hostroot\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510355 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4061c19f-9533-4feb-bcf2-3f42c037565e-os-release\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-cni-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1022377c-cbde-4f08-8540-b82161f2f16a-cni-binary-copy\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0f5b59c-9501-4223-abfd-30dad4473397-ovn-node-metrics-cert\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510450 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-multus-cni-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510462 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-tuned\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510544 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1022377c-cbde-4f08-8540-b82161f2f16a-system-cni-dir\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0f5b59c-9501-4223-abfd-30dad4473397-var-lib-openvswitch\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510757 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fa191c78-a11e-4519-a32b-c036c00c12cf-serviceca\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.510960 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0f5b59c-9501-4223-abfd-30dad4473397-ovnkube-script-lib\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.511007 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7-konnectivity-ca\") pod \"konnectivity-agent-bkvh6\" (UID: \"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7\") " pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.511031 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1022377c-cbde-4f08-8540-b82161f2f16a-cni-binary-copy\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.511660 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-tmp\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.613127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.511710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4061c19f-9533-4feb-bcf2-3f42c037565e-cni-binary-copy\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.511982 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7-agent-certs\") pod \"konnectivity-agent-bkvh6\" (UID: \"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7\") " pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.515570 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-etc-tuned\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.515622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0f5b59c-9501-4223-abfd-30dad4473397-ovn-node-metrics-cert\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.517594 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l5v4\" (UniqueName: \"kubernetes.io/projected/d5e9ba6f-bce6-455e-b018-853d30cf3a8f-kube-api-access-7l5v4\") pod \"aws-ebs-csi-driver-node-9t548\" (UID: \"d5e9ba6f-bce6-455e-b018-853d30cf3a8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.519534 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nnjh\" (UniqueName: \"kubernetes.io/projected/fa191c78-a11e-4519-a32b-c036c00c12cf-kube-api-access-8nnjh\") pod \"node-ca-pkh94\" (UID: \"fa191c78-a11e-4519-a32b-c036c00c12cf\") " pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.522428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfxq7\" (UniqueName: \"kubernetes.io/projected/3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847-kube-api-access-hfxq7\") pod \"iptables-alerter-tf9rh\" (UID: \"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847\") " pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.523124 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8fll\" (UniqueName: \"kubernetes.io/projected/06c1ae6a-23e4-4530-8071-496c47f2f077-kube-api-access-w8fll\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.524447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpzm4\" (UniqueName: \"kubernetes.io/projected/0b4404b1-b307-4b37-b215-b250669247b7-kube-api-access-xpzm4\") pod \"node-resolver-fmnt7\" (UID: \"0b4404b1-b307-4b37-b215-b250669247b7\") " pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.524502 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.524515 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.524524 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:18.524570 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:19.024555764 +0000 UTC m=+3.053114083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.525431 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctcrb\" (UniqueName: \"kubernetes.io/projected/4061c19f-9533-4feb-bcf2-3f42c037565e-kube-api-access-ctcrb\") pod \"multus-additional-cni-plugins-8l58j\" (UID: \"4061c19f-9533-4feb-bcf2-3f42c037565e\") " pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.526374 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdcmq\" (UniqueName: \"kubernetes.io/projected/bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317-kube-api-access-xdcmq\") pod \"tuned-gfdbr\" (UID: \"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317\") " pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.527360 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5hsq\" (UniqueName: \"kubernetes.io/projected/c0f5b59c-9501-4223-abfd-30dad4473397-kube-api-access-m5hsq\") pod \"ovnkube-node-nmg8g\" (UID: \"c0f5b59c-9501-4223-abfd-30dad4473397\") " pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.528607 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gzb6\" (UniqueName: \"kubernetes.io/projected/1022377c-cbde-4f08-8540-b82161f2f16a-kube-api-access-8gzb6\") pod \"multus-hmn6n\" (UID: \"1022377c-cbde-4f08-8540-b82161f2f16a\") " pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.613855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.547217 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" event={"ID":"5251b4cf7c0a51ecb6428df168d891e3","Type":"ContainerStarted","Data":"1899a9a5abb5dadb490ba59233ea0e243f0058ca3cf2cf8ef3e05dc4dbb5a1cf"} Jul 13 18:58:18.614390 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.548203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" event={"ID":"0c05c575dd3a415123dd60cca6ef09a9","Type":"ContainerStarted","Data":"9fd4522aeb4819dbfade2c79375fa53c061257f5514e1d5882ff603509049e4d"} Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.690337 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.698755 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:18.698986 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5e9ba6f_bce6_455e_b018_853d30cf3a8f.slice/crio-6c14ea3fecd57544be7385432ddc480cedae781694a7c130cb4b2264ec2c5a81 WatchSource:0}: Error finding container 6c14ea3fecd57544be7385432ddc480cedae781694a7c130cb4b2264ec2c5a81: Status 404 returned error can't find the container with id 6c14ea3fecd57544be7385432ddc480cedae781694a7c130cb4b2264ec2c5a81 Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.701278 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.706066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fmnt7" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.711271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.717258 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.722825 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8l58j" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:18.729710 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0f5b59c_9501_4223_abfd_30dad4473397.slice/crio-c33820e684ff2f71a6b8f924be755dc9d2f0330b03a41893c0a7884e542fe4b2 WatchSource:0}: Error finding container c33820e684ff2f71a6b8f924be755dc9d2f0330b03a41893c0a7884e542fe4b2: Status 404 returned error can't find the container with id c33820e684ff2f71a6b8f924be755dc9d2f0330b03a41893c0a7884e542fe4b2 Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:18.731589 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4061c19f_9533_4feb_bcf2_3f42c037565e.slice/crio-301e50786ffabcaa709152d3053d7a25a2c5c20462f3e639c23abe97d3799755 WatchSource:0}: Error finding container 301e50786ffabcaa709152d3053d7a25a2c5c20462f3e639c23abe97d3799755: Status 404 returned error can't find the container with id 301e50786ffabcaa709152d3053d7a25a2c5c20462f3e639c23abe97d3799755 Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.732600 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hmn6n" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.743735 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pkh94" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:18.749023 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tf9rh" Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:18.751846 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa191c78_a11e_4519_a32b_c036c00c12cf.slice/crio-1f25299c78430854ca381c9b7dd8929db2bbc334d2ed36f87b74764271fbfba9 WatchSource:0}: Error finding container 1f25299c78430854ca381c9b7dd8929db2bbc334d2ed36f87b74764271fbfba9: Status 404 returned error can't find the container with id 1f25299c78430854ca381c9b7dd8929db2bbc334d2ed36f87b74764271fbfba9 Jul 13 18:58:18.777736 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:18.760189 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ba7c1e1_3bfc_4a89_b3b6_5bb1663d2847.slice/crio-8fdadaaeb0423997b5ba7be028c14ee0dba552df44e9155e902d653a962ff6df WatchSource:0}: Error finding container 8fdadaaeb0423997b5ba7be028c14ee0dba552df44e9155e902d653a962ff6df: Status 404 returned error can't find the container with id 8fdadaaeb0423997b5ba7be028c14ee0dba552df44e9155e902d653a962ff6df Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.014296 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.014340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.014453 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.014453 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.014514 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:20.014498696 +0000 UTC m=+4.043057016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.014529 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:58:20.014521778 +0000 UTC m=+4.043080093 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.115422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.115594 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.115620 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.115633 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:19.121498 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.115703 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:20.115686967 +0000 UTC m=+4.144245298 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:19.312106 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.311715 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 18:58:19.437522 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.437397 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:53:18 +0000 UTC" deadline="2027-12-28 07:27:36.52862126 +0000 UTC" Jul 13 18:58:19.437522 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.437452 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12780h29m17.091174351s" Jul 13 18:58:19.540910 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.540872 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:19.541617 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.541008 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:19.541617 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.541410 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:19.541617 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.541506 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:19.541617 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.541577 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:19.541868 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:19.541666 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:19.560619 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.560569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerStarted","Data":"301e50786ffabcaa709152d3053d7a25a2c5c20462f3e639c23abe97d3799755"} Jul 13 18:58:19.565292 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.565147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fmnt7" event={"ID":"0b4404b1-b307-4b37-b215-b250669247b7","Type":"ContainerStarted","Data":"6626529450d57466ce3be3911b0c2a0e39fd32a1387b5eedbd91ef03f51768cb"} Jul 13 18:58:19.568383 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.568291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" event={"ID":"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317","Type":"ContainerStarted","Data":"fd51b16b3179b78839ebfb297e087f63420bf085695b27776500c6ea9d4bc4d8"} Jul 13 18:58:19.571341 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.571239 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pkh94" event={"ID":"fa191c78-a11e-4519-a32b-c036c00c12cf","Type":"ContainerStarted","Data":"1f25299c78430854ca381c9b7dd8929db2bbc334d2ed36f87b74764271fbfba9"} Jul 13 18:58:19.576730 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.576569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"c33820e684ff2f71a6b8f924be755dc9d2f0330b03a41893c0a7884e542fe4b2"} Jul 13 18:58:19.581367 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.581254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bkvh6" event={"ID":"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7","Type":"ContainerStarted","Data":"5b5c6ac8ca7620250766ebf74d469b50ce8aac7a7eebb650cac1e3d90368cc45"} Jul 13 18:58:19.591212 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.591168 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" event={"ID":"d5e9ba6f-bce6-455e-b018-853d30cf3a8f","Type":"ContainerStarted","Data":"6c14ea3fecd57544be7385432ddc480cedae781694a7c130cb4b2264ec2c5a81"} Jul 13 18:58:19.601506 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.601468 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tf9rh" event={"ID":"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847","Type":"ContainerStarted","Data":"8fdadaaeb0423997b5ba7be028c14ee0dba552df44e9155e902d653a962ff6df"} Jul 13 18:58:19.614150 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:19.614110 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hmn6n" event={"ID":"1022377c-cbde-4f08-8540-b82161f2f16a","Type":"ContainerStarted","Data":"2871975ad25f126e10f4b4c632f7dd3e667f98531a7484a2ac74485e011761dd"} Jul 13 18:58:20.022372 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:20.022273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:20.022561 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:20.022391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:20.022561 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.022527 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:20.022710 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.022600 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:58:22.022581565 +0000 UTC m=+6.051139901 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:20.023087 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.023066 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:20.023165 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.023126 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:22.023109441 +0000 UTC m=+6.051667760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:20.123533 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:20.123425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:20.123732 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.123647 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:20.123732 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.123691 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:20.123732 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.123708 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:20.123925 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:20.123771 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:22.123749361 +0000 UTC m=+6.152307682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:20.437838 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:20.437715 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 18:53:18 +0000 UTC" deadline="2028-02-17 18:29:00.348594667 +0000 UTC" Jul 13 18:58:20.437838 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:20.437782 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14015h30m39.910817037s" Jul 13 18:58:21.541199 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:21.541114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:21.541636 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:21.541283 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:21.541736 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:21.541696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:21.541937 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:21.541792 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:21.541937 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:21.541856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:21.541937 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:21.541923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:22.041092 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:22.041050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:22.041289 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:22.041116 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:22.041289 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.041249 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:22.041402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.041315 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:26.041295731 +0000 UTC m=+10.069854063 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:22.041817 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.041760 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:22.041817 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.041815 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:58:26.041799105 +0000 UTC m=+10.070357421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:22.141910 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:22.141860 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:22.142134 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.142107 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:22.142134 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.142133 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:22.142252 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.142147 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:22.142252 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:22.142221 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:26.142200687 +0000 UTC m=+10.170759024 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:23.541163 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:23.541066 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:23.541163 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:23.541113 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:23.541748 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:23.541175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:23.541748 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:23.541193 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:23.541748 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:23.541291 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:23.541748 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:23.541390 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:25.542185 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:25.541293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:25.542185 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:25.541428 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:25.542185 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:25.541888 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:25.542185 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:25.541978 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:25.542185 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:25.542039 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:25.542185 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:25.542120 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:26.077338 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:26.077287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:26.077524 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:26.077348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:26.077524 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.077470 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:26.077524 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.077471 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:26.077742 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.077548 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:34.077528984 +0000 UTC m=+18.106087312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:26.077742 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.077566 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:58:34.077557711 +0000 UTC m=+18.106116027 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:26.179327 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:26.178616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:26.179327 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.178849 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:26.179327 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.178876 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:26.179327 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.178889 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:26.179327 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:26.178954 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:34.178931567 +0000 UTC m=+18.207489893 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:27.541190 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:27.540653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:27.541190 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:27.540725 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:27.541190 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:27.540830 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:27.541190 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:27.540889 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:27.541190 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:27.540913 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:27.541190 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:27.541005 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:29.541047 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:29.540999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:29.541492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:29.541004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:29.541492 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:29.541142 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:29.541492 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:29.541147 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:29.541492 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:29.541249 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:29.541492 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:29.541343 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:31.540996 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:31.540959 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:31.540996 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:31.540995 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:31.541486 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:31.541000 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:31.541486 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:31.541078 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:31.541486 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:31.541163 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:31.541486 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:31.541245 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:33.541072 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:33.541030 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:33.541072 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:33.541069 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:33.541612 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:33.541029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:33.541612 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:33.541155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:33.541612 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:33.541208 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:33.541612 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:33.541327 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:34.138145 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:34.138115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:34.138304 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:34.138268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:34.138304 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.138292 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:34.138421 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.138353 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:34.138421 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.138363 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:50.138348321 +0000 UTC m=+34.166906636 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:34.138421 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.138396 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:58:50.138382532 +0000 UTC m=+34.166940864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:34.239634 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:34.239591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:34.239828 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.239774 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:34.239828 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.239801 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:34.239828 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.239816 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:34.239995 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:34.239889 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:50.239874706 +0000 UTC m=+34.268433020 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:35.541179 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:35.541135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:35.541702 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:35.541135 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:35.541702 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:35.541287 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:35.541702 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:35.541388 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:35.541702 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:35.541145 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:35.541702 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:35.541494 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:37.541542 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:37.541503 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:37.542079 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:37.541550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:37.542079 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:37.541550 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:37.542079 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:37.541638 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:37.542079 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:37.541766 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:37.542079 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:37.541877 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:38.663414 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.663180 2580 generic.go:356] "Generic (PLEG): container finished" podID="5251b4cf7c0a51ecb6428df168d891e3" containerID="790261ab0abdd8a16e8de1cd0128b7ff44a1e75ec5c05afb8579fdc928c08b6b" exitCode=0 Jul 13 18:58:38.664302 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.663704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" event={"ID":"5251b4cf7c0a51ecb6428df168d891e3","Type":"ContainerDied","Data":"790261ab0abdd8a16e8de1cd0128b7ff44a1e75ec5c05afb8579fdc928c08b6b"} Jul 13 18:58:38.668819 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.668771 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" event={"ID":"0c05c575dd3a415123dd60cca6ef09a9","Type":"ContainerStarted","Data":"ed315918962ae1d5a3e9b77399b2b8839d9f38b994dc511fbb987c5c3e5bde69"} Jul 13 18:58:38.672775 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.672703 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hmn6n" event={"ID":"1022377c-cbde-4f08-8540-b82161f2f16a","Type":"ContainerStarted","Data":"129b1f34834048fa8b12139348033a04ff442a2c3b93fd34ea1ea755c2ad5638"} Jul 13 18:58:38.674412 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.674385 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerStarted","Data":"712c02e2c6dd0e45cf90af9536e11c1f81f6492b3ece385f8b9a6e6ef85b6b75"} Jul 13 18:58:38.675930 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.675901 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fmnt7" event={"ID":"0b4404b1-b307-4b37-b215-b250669247b7","Type":"ContainerStarted","Data":"335cef72798a807a5114cc464b548c1a0895b79135935cc4f15f5d0dd6e48bea"} Jul 13 18:58:38.677777 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.677752 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" event={"ID":"bbdff4b4-a97f-4c80-b8a7-acfa6f5fe317","Type":"ContainerStarted","Data":"b6b5a304da228207013cbc85da786d031bba09183e99b318c574b0f168edf055"} Jul 13 18:58:38.679139 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.679114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pkh94" event={"ID":"fa191c78-a11e-4519-a32b-c036c00c12cf","Type":"ContainerStarted","Data":"0245b0a9091c9f4da1936d24e6f669400fe48b4161692b93832c8b556e7090ad"} Jul 13 18:58:38.681006 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.680970 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 18:58:38.681302 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.681281 2580 generic.go:356] "Generic (PLEG): container finished" podID="c0f5b59c-9501-4223-abfd-30dad4473397" containerID="7bbe7949254030bca0efad7e3a03547d7e54c37e197a94308bab3b7fef00f361" exitCode=1 Jul 13 18:58:38.681369 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.681349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"a1bd545d73808d8aae6cebceb3581a3294d05394285328d495b00b5d613afa48"} Jul 13 18:58:38.681421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.681401 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerDied","Data":"7bbe7949254030bca0efad7e3a03547d7e54c37e197a94308bab3b7fef00f361"} Jul 13 18:58:38.681469 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.681431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"9de1bbff7b49234cba58ba9a63c412e5fa0c60fdd52ca88bc1c598b7cd4f59e6"} Jul 13 18:58:38.682958 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.682899 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bkvh6" event={"ID":"9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7","Type":"ContainerStarted","Data":"7b86507cb64381e768200c3e324bc582051560caff59e6858829ae07467fafd1"} Jul 13 18:58:38.684249 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.684220 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" event={"ID":"d5e9ba6f-bce6-455e-b018-853d30cf3a8f","Type":"ContainerStarted","Data":"b3a91353aa08d1c8438532d4af7fb761b3010b65c2d8fda1f54d9f27ac14a38a"} Jul 13 18:58:38.701421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.701371 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hmn6n" podStartSLOduration=3.197993303 podStartE2EDuration="22.701350585s" totalImagesPullingTime="19.503357282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:38.701303935 +0000 UTC m=+22.729862272" watchObservedRunningTime="2026-07-13 18:58:38.701350585 +0000 UTC m=+22.729908917" Jul 13 18:58:38.734166 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.734105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fmnt7" podStartSLOduration=3.204891123 podStartE2EDuration="22.734083786s" totalImagesPullingTime="19.529192663s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:38.716839201 +0000 UTC m=+22.745397564" watchObservedRunningTime="2026-07-13 18:58:38.734083786 +0000 UTC m=+22.762642124" Jul 13 18:58:38.755544 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.755502 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-70.ec2.internal" podStartSLOduration=21.755487045 podStartE2EDuration="21.755487045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:38.733570839 +0000 UTC m=+22.762129204" watchObservedRunningTime="2026-07-13 18:58:38.755487045 +0000 UTC m=+22.784045381" Jul 13 18:58:38.769775 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.769732 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bkvh6" podStartSLOduration=7.391242827 podStartE2EDuration="22.769715206s" totalImagesPullingTime="15.378472379s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:38.769169366 +0000 UTC m=+22.797727704" watchObservedRunningTime="2026-07-13 18:58:38.769715206 +0000 UTC m=+22.798273543" Jul 13 18:58:38.787372 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.787254 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pkh94" podStartSLOduration=2.303276371 podStartE2EDuration="21.787232191s" totalImagesPullingTime="19.48395582s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:38.786577081 +0000 UTC m=+22.815135438" watchObservedRunningTime="2026-07-13 18:58:38.787232191 +0000 UTC m=+22.815790530" Jul 13 18:58:38.810722 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:38.810657 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gfdbr" podStartSLOduration=3.2750258150000002 podStartE2EDuration="22.810639602s" totalImagesPullingTime="19.535613787s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:38.810149279 +0000 UTC m=+22.838707615" watchObservedRunningTime="2026-07-13 18:58:38.810639602 +0000 UTC m=+22.839197938" Jul 13 18:58:39.541319 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.541225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:39.541556 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.541349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:39.541556 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:39.541363 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:39.541556 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.541379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:39.541556 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:39.541447 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:39.541556 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:39.541511 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:39.690237 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.690197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" event={"ID":"5251b4cf7c0a51ecb6428df168d891e3","Type":"ContainerStarted","Data":"4b20b0c589d800d0855fcebca72d8d2d25bcd156479ce8858a8bf227c613690f"} Jul 13 18:58:39.691764 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.691735 2580 generic.go:356] "Generic (PLEG): container finished" podID="4061c19f-9533-4feb-bcf2-3f42c037565e" containerID="712c02e2c6dd0e45cf90af9536e11c1f81f6492b3ece385f8b9a6e6ef85b6b75" exitCode=0 Jul 13 18:58:39.691846 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.691818 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerDied","Data":"712c02e2c6dd0e45cf90af9536e11c1f81f6492b3ece385f8b9a6e6ef85b6b75"} Jul 13 18:58:39.694825 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.694808 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 18:58:39.695307 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.695282 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"6d0528d2cd7e85c934aedff01dc50af2673e1403f9f99921109f9c812b0e0bdd"} Jul 13 18:58:39.695380 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.695354 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"dd0ae077fbd7210696f7c8aa2ccd1a0bbc9b5ebe78706b9b090181c40130cff5"} Jul 13 18:58:39.695427 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.695388 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"3cf5cb61d4fc48d9f10cab49cb09e4d8028d10f7a7cfd89857b0cb8bca7ee79d"} Jul 13 18:58:39.705368 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:39.705319 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-70.ec2.internal" podStartSLOduration=22.705304076 podStartE2EDuration="22.705304076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:39.704987505 +0000 UTC m=+23.733545861" watchObservedRunningTime="2026-07-13 18:58:39.705304076 +0000 UTC m=+23.733862413" Jul 13 18:58:40.627742 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.627716 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 18:58:40.641842 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.641733 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T18:58:40.627737436Z","UUID":"b8474d72-29f9-4b1c-9e29-e5b7c2e4d92c","Handler":null,"Name":"","Endpoint":""} Jul 13 18:58:40.643639 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.643615 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 18:58:40.643782 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.643658 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 18:58:40.699402 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.699366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tf9rh" event={"ID":"3ba7c1e1-3bfc-4a89-b3b6-5bb1663d2847","Type":"ContainerStarted","Data":"8da0338ace26e0f7db7afd74633f65d65168edb19d34ea620bd0f9c7a862d197"} Jul 13 18:58:40.701633 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.701597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" event={"ID":"d5e9ba6f-bce6-455e-b018-853d30cf3a8f","Type":"ContainerStarted","Data":"83faafaa1730ffca78c197cd4710e30e6d1575c06ec383b90b1995509c0fb30d"} Jul 13 18:58:40.716236 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:40.716168 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tf9rh" podStartSLOduration=4.232197978 podStartE2EDuration="23.716145481s" totalImagesPullingTime="19.483947503s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:40.715883771 +0000 UTC m=+24.744442107" watchObservedRunningTime="2026-07-13 18:58:40.716145481 +0000 UTC m=+24.744703835" Jul 13 18:58:41.540899 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.540842 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:41.541061 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:41.540971 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:41.541131 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.541059 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:41.541203 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:41.541180 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:41.541261 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.541226 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:41.541348 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:41.541327 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:41.706706 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.706646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" event={"ID":"d5e9ba6f-bce6-455e-b018-853d30cf3a8f","Type":"ContainerStarted","Data":"cd8a25f2babf20bfd5543d1dff15d56bc750d41beaad7fe0625aef6f4b327916"} Jul 13 18:58:41.710294 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.710266 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 18:58:41.710740 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.710708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"e77d7577cce2114a7e7ed0b868f39e169cf290e633c957e3334371dfc36b0089"} Jul 13 18:58:41.725353 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:41.725300 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9t548" podStartSLOduration=3.429640448 podStartE2EDuration="25.725282181s" totalImagesPullingTime="22.295641733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:41.72488213 +0000 UTC m=+25.753440485" watchObservedRunningTime="2026-07-13 18:58:41.725282181 +0000 UTC m=+25.753840518" Jul 13 18:58:42.910047 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:42.909802 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:42.910663 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:42.910536 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:43.541410 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:43.541376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:43.541410 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:43.541398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:43.541709 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:43.541383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:43.541709 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:43.541500 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:43.541709 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:43.541551 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:43.541709 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:43.541633 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:43.715702 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:43.715653 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:43.716264 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:43.716240 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bkvh6" Jul 13 18:58:44.722245 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.721970 2580 generic.go:356] "Generic (PLEG): container finished" podID="4061c19f-9533-4feb-bcf2-3f42c037565e" containerID="7a1170935c47a258a5d52e2ad5730a6444d7781bfb1a69feb62d0b2d2c944441" exitCode=0 Jul 13 18:58:44.723047 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.722051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerDied","Data":"7a1170935c47a258a5d52e2ad5730a6444d7781bfb1a69feb62d0b2d2c944441"} Jul 13 18:58:44.728227 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.728206 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 18:58:44.728615 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.728584 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"ef1fd9678648162c53ee543921182ed71ae32d81b071797f11278d6d0c574954"} Jul 13 18:58:44.728913 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.728897 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:44.728996 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.728968 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:44.729093 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.729077 2580 scope.go:117] "RemoveContainer" containerID="7bbe7949254030bca0efad7e3a03547d7e54c37e197a94308bab3b7fef00f361" Jul 13 18:58:44.745441 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.745411 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:44.745579 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:44.745556 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:45.540839 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.540795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:45.541133 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.540797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:45.541133 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.540929 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:45.541133 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:45.540925 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:45.541133 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:45.541018 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:45.541133 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:45.541080 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:45.734011 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.733983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 18:58:45.734424 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.734385 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" event={"ID":"c0f5b59c-9501-4223-abfd-30dad4473397","Type":"ContainerStarted","Data":"32af3bbd80e7b504110e79e5d628280346b59206084c2c572cad6756043480ea"} Jul 13 18:58:45.734494 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.734451 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 18:58:45.763286 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:45.763236 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" podStartSLOduration=10.160177689 podStartE2EDuration="29.763216543s" totalImagesPullingTime="19.603038854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:58:45.762828508 +0000 UTC m=+29.791386864" watchObservedRunningTime="2026-07-13 18:58:45.763216543 +0000 UTC m=+29.791774880" Jul 13 18:58:46.095166 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.095129 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:58:46.415300 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.415265 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zsgck"] Jul 13 18:58:46.415478 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.415379 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:46.415519 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:46.415482 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:46.417802 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.417775 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bpzhc"] Jul 13 18:58:46.417942 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.417914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:46.418062 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:46.418041 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:46.421223 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.421198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vk9kv"] Jul 13 18:58:46.421342 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.421299 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:46.421414 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:46.421394 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:46.738768 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.738662 2580 generic.go:356] "Generic (PLEG): container finished" podID="4061c19f-9533-4feb-bcf2-3f42c037565e" containerID="157567ea49396169e298a9c82196b22a8b32fde2f671199ab38aba77b659905a" exitCode=0 Jul 13 18:58:46.738768 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:46.738704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerDied","Data":"157567ea49396169e298a9c82196b22a8b32fde2f671199ab38aba77b659905a"} Jul 13 18:58:47.540607 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:47.540518 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:47.540779 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:47.540643 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:48.541277 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:48.541046 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:48.541631 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:48.541045 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:48.541631 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:48.541304 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:48.541631 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:48.541384 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:48.745972 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:48.745935 2580 generic.go:356] "Generic (PLEG): container finished" podID="4061c19f-9533-4feb-bcf2-3f42c037565e" containerID="fa4b6f1c7bfa18a62e60cdb39dc51c827f4109b9324848815baff8aa9bbf65a5" exitCode=0 Jul 13 18:58:48.746154 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:48.746027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerDied","Data":"fa4b6f1c7bfa18a62e60cdb39dc51c827f4109b9324848815baff8aa9bbf65a5"} Jul 13 18:58:49.541366 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:49.541332 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:49.541774 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:49.541440 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:50.150090 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:50.150005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:50.150090 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:50.150051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:50.150334 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.150168 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:50.150334 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.150206 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:50.150334 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.150249 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret podName:8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa nodeName:}" failed. No retries permitted until 2026-07-13 18:59:22.150228384 +0000 UTC m=+66.178786702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret") pod "global-pull-secret-syncer-zsgck" (UID: "8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa") : object "kube-system"/"original-pull-secret" not registered Jul 13 18:58:50.150334 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.150271 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs podName:06c1ae6a-23e4-4530-8071-496c47f2f077 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:22.150260279 +0000 UTC m=+66.178818594 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs") pod "network-metrics-daemon-bpzhc" (UID: "06c1ae6a-23e4-4530-8071-496c47f2f077") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 18:58:50.250721 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:50.250683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:50.250904 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.250881 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 18:58:50.250956 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.250912 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 18:58:50.250956 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.250926 2580 projected.go:196] Error preparing data for projected volume kube-api-access-lnrn6 for pod openshift-network-diagnostics/network-check-target-vk9kv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:50.251042 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.250994 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6 podName:3fa7146c-160b-4480-9755-33c605de0107 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:22.250973681 +0000 UTC m=+66.279532003 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lnrn6" (UniqueName: "kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6") pod "network-check-target-vk9kv" (UID: "3fa7146c-160b-4480-9755-33c605de0107") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 18:58:50.541523 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:50.541440 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:50.542202 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.541592 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-bpzhc" podUID="06c1ae6a-23e4-4530-8071-496c47f2f077" Jul 13 18:58:50.542202 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:50.541653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:50.542202 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:50.541783 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vk9kv" podUID="3fa7146c-160b-4480-9755-33c605de0107" Jul 13 18:58:51.540650 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:51.540608 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:51.540845 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:51.540751 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zsgck" podUID="8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa" Jul 13 18:58:52.229148 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.229119 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-70.ec2.internal" event="NodeReady" Jul 13 18:58:52.229708 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.229267 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 18:58:52.270205 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.270169 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fb8548b8f-9swm6"] Jul 13 18:58:52.275226 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.275198 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.277802 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.277777 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 18:58:52.277961 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.277783 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 18:58:52.278282 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.278264 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 18:58:52.279116 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.279100 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xpzcf\"" Jul 13 18:58:52.286067 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.286047 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 18:58:52.287134 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.286385 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch"] Jul 13 18:58:52.289760 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.289741 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-44q2g"] Jul 13 18:58:52.289871 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.289853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.292617 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.292364 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.293006 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.292987 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8"] Jul 13 18:58:52.293119 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.293105 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.295000 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.294818 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 13 18:58:52.296472 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.296429 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-57897d6b58-6nnn2"] Jul 13 18:58:52.296718 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.296660 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 13 18:58:52.297474 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.296848 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" Jul 13 18:58:52.297474 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.297026 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.297474 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.297268 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.297474 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.297328 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d9lm6\"" Jul 13 18:58:52.298877 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.297927 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.298877 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.298047 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 13 18:58:52.300660 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.300640 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb8548b8f-9swm6"] Jul 13 18:58:52.301753 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.300984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.301753 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.301324 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 13 18:58:52.301753 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.300663 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-q62wv\"" Jul 13 18:58:52.302031 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.301923 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cd7k9\"" Jul 13 18:58:52.302505 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.302347 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.302505 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.302392 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.303375 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.303321 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch"] Jul 13 18:58:52.303880 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.303861 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 13 18:58:52.305083 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.304934 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.306481 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.306206 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.306481 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.306217 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 13 18:58:52.306481 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.306210 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 13 18:58:52.306705 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.306618 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-67jdm\"" Jul 13 18:58:52.307158 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.306921 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 13 18:58:52.316125 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.316090 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 13 18:58:52.316316 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.316215 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-44q2g"] Jul 13 18:58:52.317263 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.317236 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59"] Jul 13 18:58:52.321466 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.321437 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft"] Jul 13 18:58:52.321615 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.321597 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.324840 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.324803 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-kh496"] Jul 13 18:58:52.324973 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.324931 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:52.325500 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.325478 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.325798 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.325777 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 13 18:58:52.325798 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.325802 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-d4zfl\"" Jul 13 18:58:52.326017 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.325915 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 13 18:58:52.326494 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.326230 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.327961 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.327942 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57897d6b58-6nnn2"] Jul 13 18:58:52.328043 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.327969 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8v42n"] Jul 13 18:58:52.328096 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.328068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.329455 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.329304 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.329569 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.329529 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.329897 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.329770 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r6xgd\"" Jul 13 18:58:52.330049 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.329980 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 13 18:58:52.330659 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.330628 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 13 18:58:52.332050 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.332034 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.332774 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.332754 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk"] Jul 13 18:58:52.333414 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.332783 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 13 18:58:52.333414 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.332940 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-dc89n\"" Jul 13 18:58:52.333558 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.332922 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.333654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.332997 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:52.336807 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.336787 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bk62w"] Jul 13 18:58:52.337879 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.337855 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" Jul 13 18:58:52.337995 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.337956 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 13 18:58:52.338625 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.338607 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 13 18:58:52.339817 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.339797 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qkrht\"" Jul 13 18:58:52.339968 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.339949 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h"] Jul 13 18:58:52.340138 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.340122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.340919 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.340898 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 13 18:58:52.341368 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.341348 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.341566 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.341546 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4tl4d\"" Jul 13 18:58:52.341899 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.341873 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.342824 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.342805 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.343127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343106 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8v42n"] Jul 13 18:58:52.343127 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343120 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.343269 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343139 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59"] Jul 13 18:58:52.343269 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343183 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8"] Jul 13 18:58:52.343269 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343201 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bk62w"] Jul 13 18:58:52.343269 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343219 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kh496"] Jul 13 18:58:52.343269 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343237 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk"] Jul 13 18:58:52.343269 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343255 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft"] Jul 13 18:58:52.343539 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.343342 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.344431 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.344413 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mbvx8\"" Jul 13 18:58:52.344563 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.344545 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 18:58:52.345937 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.345916 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 13 18:58:52.346389 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.346369 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 13 18:58:52.346389 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.346383 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 13 18:58:52.346751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.346733 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-866w2\"" Jul 13 18:58:52.348356 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.348308 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 13 18:58:52.349649 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.349631 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h"] Jul 13 18:58:52.366965 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.366933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-installation-pull-secrets\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367108 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367039 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrj6h\" (UniqueName: \"kubernetes.io/projected/861e258d-9123-4ad0-8e74-d3b682a1746f-kube-api-access-qrj6h\") pod \"volume-data-source-validator-78495597f6-7vrr8\" (UID: \"861e258d-9123-4ad0-8e74-d3b682a1746f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" Jul 13 18:58:52.367108 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc33eccb-8c33-46d3-8c49-87ce85e050af-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.367216 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367154 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8dac37f-f511-4f56-9746-827c3213809c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.367216 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.367216 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-stats-auth\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.367364 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcld5\" (UniqueName: \"kubernetes.io/projected/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-kube-api-access-hcld5\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.367364 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367281 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-registry-certificates\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367364 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-bound-sa-token\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367364 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367351 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc33eccb-8c33-46d3-8c49-87ce85e050af-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.367513 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367380 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-default-certificate\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.367513 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.367513 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-trusted-ca\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367513 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc33eccb-8c33-46d3-8c49-87ce85e050af-snapshots\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.367513 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367487 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltr4t\" (UniqueName: \"kubernetes.io/projected/e8dac37f-f511-4f56-9746-827c3213809c-kube-api-access-ltr4t\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.367745 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367745 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367542 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73b27864-68b8-4b8f-84b2-47dc93109879-ca-trust-extracted\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367745 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc33eccb-8c33-46d3-8c49-87ce85e050af-serving-cert\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.367864 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367641 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.367864 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367783 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-image-registry-private-configuration\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367864 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrcck\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-kube-api-access-mrcck\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.367864 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc33eccb-8c33-46d3-8c49-87ce85e050af-tmp\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.368040 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.367881 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnmsp\" (UniqueName: \"kubernetes.io/projected/fc33eccb-8c33-46d3-8c49-87ce85e050af-kube-api-access-mnmsp\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.383392 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.383351 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vf6p7"] Jul 13 18:58:52.387266 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.387241 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.390113 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.390086 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wt7nz\"" Jul 13 18:58:52.390261 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.390133 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 18:58:52.390261 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.390148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 18:58:52.399110 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.399070 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vf6p7"] Jul 13 18:58:52.468994 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.468962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-default-certificate\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.468994 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469007 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:52.469245 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-tmp-dir\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.469245 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x64gd\" (UniqueName: \"kubernetes.io/projected/67985222-8a17-45f7-b42c-fa011cdd7fd1-kube-api-access-x64gd\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.469245 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469192 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:52.469404 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73b27864-68b8-4b8f-84b2-47dc93109879-ca-trust-extracted\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.469404 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkjxv\" (UniqueName: \"kubernetes.io/projected/84708eb3-266e-43f3-904a-8c038080bb47-kube-api-access-mkjxv\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:52.469404 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-image-registry-private-configuration\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.469554 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc33eccb-8c33-46d3-8c49-87ce85e050af-tmp\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.469554 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67985222-8a17-45f7-b42c-fa011cdd7fd1-config\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.469554 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469481 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-installation-pull-secrets\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.469554 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469514 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-config-volume\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.469751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469560 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcpbm\" (UniqueName: \"kubernetes.io/projected/eaea0df7-d85f-48b1-b100-dacf528c63b2-kube-api-access-mcpbm\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.469751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrj6h\" (UniqueName: \"kubernetes.io/projected/861e258d-9123-4ad0-8e74-d3b682a1746f-kube-api-access-qrj6h\") pod \"volume-data-source-validator-78495597f6-7vrr8\" (UID: \"861e258d-9123-4ad0-8e74-d3b682a1746f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" Jul 13 18:58:52.469751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc33eccb-8c33-46d3-8c49-87ce85e050af-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.469751 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkjhs\" (UniqueName: \"kubernetes.io/projected/23b75ff0-17af-4e4f-be25-be61ce96cfab-kube-api-access-mkjhs\") pod \"ingress-canary-bk62w\" (UID: \"23b75ff0-17af-4e4f-be25-be61ce96cfab\") " pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.469944 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469748 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73b27864-68b8-4b8f-84b2-47dc93109879-ca-trust-extracted\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.469944 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-stats-auth\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.469944 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcld5\" (UniqueName: \"kubernetes.io/projected/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-kube-api-access-hcld5\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.469944 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469906 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-registry-certificates\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.469944 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-trusted-ca\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469955 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc33eccb-8c33-46d3-8c49-87ce85e050af-tmp\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.469991 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-trusted-ca\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct5xb\" (UniqueName: \"kubernetes.io/projected/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-kube-api-access-ct5xb\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac419028-3515-491f-9b60-e72e2292fea6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470104 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltr4t\" (UniqueName: \"kubernetes.io/projected/e8dac37f-f511-4f56-9746-827c3213809c-kube-api-access-ltr4t\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470128 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-serving-cert\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470156 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.470189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc33eccb-8c33-46d3-8c49-87ce85e050af-serving-cert\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnmsp\" (UniqueName: \"kubernetes.io/projected/fc33eccb-8c33-46d3-8c49-87ce85e050af-kube-api-access-mnmsp\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaea0df7-d85f-48b1-b100-dacf528c63b2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrcck\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-kube-api-access-mrcck\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67985222-8a17-45f7-b42c-fa011cdd7fd1-serving-cert\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23b75ff0-17af-4e4f-be25-be61ce96cfab-cert\") pod \"ingress-canary-bk62w\" (UID: \"23b75ff0-17af-4e4f-be25-be61ce96cfab\") " pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-config\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470425 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaea0df7-d85f-48b1-b100-dacf528c63b2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470460 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8dac37f-f511-4f56-9746-827c3213809c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wmdc\" (UniqueName: \"kubernetes.io/projected/b32a6104-ba31-44e4-adb2-07496b9fe7d2-kube-api-access-2wmdc\") pod \"network-check-source-869dcfcdcd-zh6fk\" (UID: \"b32a6104-ba31-44e4-adb2-07496b9fe7d2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470542 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc33eccb-8c33-46d3-8c49-87ce85e050af-snapshots\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-bound-sa-token\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc33eccb-8c33-46d3-8c49-87ce85e050af-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.470654 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zpqn\" (UniqueName: \"kubernetes.io/projected/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-kube-api-access-6zpqn\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.470771 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.970752321 +0000 UTC m=+36.999310658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : configmap references non-existent config key: service-ca.crt Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470775 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc33eccb-8c33-46d3-8c49-87ce85e050af-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470936 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-trusted-ca\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.470963 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.471023 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.971004555 +0000 UTC m=+36.999562880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : secret "router-metrics-certs-default" not found Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.471070 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.471085 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb8548b8f-9swm6: secret "image-registry-tls" not found Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.471137 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls podName:73b27864-68b8-4b8f-84b2-47dc93109879 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.971121221 +0000 UTC m=+36.999679536 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls") pod "image-registry-6fb8548b8f-9swm6" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879") : secret "image-registry-tls" not found Jul 13 18:58:52.471407 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.470624 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-registry-certificates\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.471881 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.471544 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc33eccb-8c33-46d3-8c49-87ce85e050af-snapshots\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.471881 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.471697 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc33eccb-8c33-46d3-8c49-87ce85e050af-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.471881 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.471807 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:52.471881 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.471873 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls podName:e8dac37f-f511-4f56-9746-827c3213809c nodeName:}" failed. No retries permitted until 2026-07-13 18:58:52.971858529 +0000 UTC m=+37.000416860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tnch" (UID: "e8dac37f-f511-4f56-9746-827c3213809c") : secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:52.472278 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.472226 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e8dac37f-f511-4f56-9746-827c3213809c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.474948 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.474923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc33eccb-8c33-46d3-8c49-87ce85e050af-serving-cert\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.474948 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.474948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-stats-auth\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.475120 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.474954 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-installation-pull-secrets\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.475120 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.474993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-default-certificate\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.475517 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.475494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-image-registry-private-configuration\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.482156 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.482075 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrj6h\" (UniqueName: \"kubernetes.io/projected/861e258d-9123-4ad0-8e74-d3b682a1746f-kube-api-access-qrj6h\") pod \"volume-data-source-validator-78495597f6-7vrr8\" (UID: \"861e258d-9123-4ad0-8e74-d3b682a1746f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" Jul 13 18:58:52.484048 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.483976 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrcck\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-kube-api-access-mrcck\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.484048 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.483993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcld5\" (UniqueName: \"kubernetes.io/projected/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-kube-api-access-hcld5\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.484563 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.484540 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnmsp\" (UniqueName: \"kubernetes.io/projected/fc33eccb-8c33-46d3-8c49-87ce85e050af-kube-api-access-mnmsp\") pod \"insights-operator-68d9dd94cf-44q2g\" (UID: \"fc33eccb-8c33-46d3-8c49-87ce85e050af\") " pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.485332 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.485308 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltr4t\" (UniqueName: \"kubernetes.io/projected/e8dac37f-f511-4f56-9746-827c3213809c-kube-api-access-ltr4t\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.486590 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.486568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-bound-sa-token\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.545908 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.545872 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:58:52.546102 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.545921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:58:52.548410 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.548381 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bqf9d\"" Jul 13 18:58:52.549066 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.548848 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 18:58:52.549066 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.548892 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvxxg\"" Jul 13 18:58:52.549334 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.549311 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:52.572042 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkjxv\" (UniqueName: \"kubernetes.io/projected/84708eb3-266e-43f3-904a-8c038080bb47-kube-api-access-mkjxv\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:52.572213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67985222-8a17-45f7-b42c-fa011cdd7fd1-config\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.572213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-config-volume\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.572213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcpbm\" (UniqueName: \"kubernetes.io/projected/eaea0df7-d85f-48b1-b100-dacf528c63b2-kube-api-access-mcpbm\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.572213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkjhs\" (UniqueName: \"kubernetes.io/projected/23b75ff0-17af-4e4f-be25-be61ce96cfab-kube-api-access-mkjhs\") pod \"ingress-canary-bk62w\" (UID: \"23b75ff0-17af-4e4f-be25-be61ce96cfab\") " pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-trusted-ca\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct5xb\" (UniqueName: \"kubernetes.io/projected/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-kube-api-access-ct5xb\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572299 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac419028-3515-491f-9b60-e72e2292fea6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-serving-cert\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572370 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaea0df7-d85f-48b1-b100-dacf528c63b2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.572421 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67985222-8a17-45f7-b42c-fa011cdd7fd1-serving-cert\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23b75ff0-17af-4e4f-be25-be61ce96cfab-cert\") pod \"ingress-canary-bk62w\" (UID: \"23b75ff0-17af-4e4f-be25-be61ce96cfab\") " pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-config\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaea0df7-d85f-48b1-b100-dacf528c63b2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wmdc\" (UniqueName: \"kubernetes.io/projected/b32a6104-ba31-44e4-adb2-07496b9fe7d2-kube-api-access-2wmdc\") pod \"network-check-source-869dcfcdcd-zh6fk\" (UID: \"b32a6104-ba31-44e4-adb2-07496b9fe7d2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zpqn\" (UniqueName: \"kubernetes.io/projected/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-kube-api-access-6zpqn\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67985222-8a17-45f7-b42c-fa011cdd7fd1-config\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572791 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-config-volume\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:52.572867 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572853 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-tmp-dir\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.572884 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.572951 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert podName:ac419028-3515-491f-9b60-e72e2292fea6 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.072933188 +0000 UTC m=+37.101491522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8v42n" (UID: "ac419028-3515-491f-9b60-e72e2292fea6") : secret "networking-console-plugin-cert" not found Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.572884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x64gd\" (UniqueName: \"kubernetes.io/projected/67985222-8a17-45f7-b42c-fa011cdd7fd1-kube-api-access-x64gd\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.573069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.573108 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.573176 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.573178 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls podName:54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.073159726 +0000 UTC m=+37.101718064 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls") pod "dns-default-vf6p7" (UID: "54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d") : secret "dns-default-metrics-tls" not found Jul 13 18:58:52.573342 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.573231 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls podName:84708eb3-266e-43f3-904a-8c038080bb47 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.073217715 +0000 UTC m=+37.101776030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls") pod "cluster-samples-operator-65db858894-hdzft" (UID: "84708eb3-266e-43f3-904a-8c038080bb47") : secret "samples-operator-tls" not found Jul 13 18:58:52.573769 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.573383 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-tmp-dir\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.573769 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.573655 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ac419028-3515-491f-9b60-e72e2292fea6-nginx-conf\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:52.573769 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.573759 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-trusted-ca\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.573912 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.573825 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-config\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.574192 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.574164 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/eaea0df7-d85f-48b1-b100-dacf528c63b2-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.575770 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.575717 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67985222-8a17-45f7-b42c-fa011cdd7fd1-serving-cert\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.576072 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.576049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-serving-cert\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.576187 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.576168 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23b75ff0-17af-4e4f-be25-be61ce96cfab-cert\") pod \"ingress-canary-bk62w\" (UID: \"23b75ff0-17af-4e4f-be25-be61ce96cfab\") " pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.576240 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.576199 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/eaea0df7-d85f-48b1-b100-dacf528c63b2-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.581987 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.581933 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcpbm\" (UniqueName: \"kubernetes.io/projected/eaea0df7-d85f-48b1-b100-dacf528c63b2-kube-api-access-mcpbm\") pod \"kube-storage-version-migrator-operator-799cb7f958-ngr59\" (UID: \"eaea0df7-d85f-48b1-b100-dacf528c63b2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.582107 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.582017 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zpqn\" (UniqueName: \"kubernetes.io/projected/5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e-kube-api-access-6zpqn\") pod \"console-operator-7866497494-kh496\" (UID: \"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e\") " pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.583517 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.583470 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkjxv\" (UniqueName: \"kubernetes.io/projected/84708eb3-266e-43f3-904a-8c038080bb47-kube-api-access-mkjxv\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:52.584225 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.584156 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x64gd\" (UniqueName: \"kubernetes.io/projected/67985222-8a17-45f7-b42c-fa011cdd7fd1-kube-api-access-x64gd\") pod \"service-ca-operator-7695fc6c8c-sw65h\" (UID: \"67985222-8a17-45f7-b42c-fa011cdd7fd1\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.584617 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.584577 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct5xb\" (UniqueName: \"kubernetes.io/projected/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-kube-api-access-ct5xb\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:52.585193 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.585145 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wmdc\" (UniqueName: \"kubernetes.io/projected/b32a6104-ba31-44e4-adb2-07496b9fe7d2-kube-api-access-2wmdc\") pod \"network-check-source-869dcfcdcd-zh6fk\" (UID: \"b32a6104-ba31-44e4-adb2-07496b9fe7d2\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" Jul 13 18:58:52.585378 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.585333 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkjhs\" (UniqueName: \"kubernetes.io/projected/23b75ff0-17af-4e4f-be25-be61ce96cfab-kube-api-access-mkjhs\") pod \"ingress-canary-bk62w\" (UID: \"23b75ff0-17af-4e4f-be25-be61ce96cfab\") " pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.614685 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.614636 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" Jul 13 18:58:52.625627 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.625595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" Jul 13 18:58:52.641250 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.641218 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:52.643952 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.643934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" Jul 13 18:58:52.662115 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.662078 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:58:52.679185 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.679151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" Jul 13 18:58:52.687168 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.687145 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bk62w" Jul 13 18:58:52.695991 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.695965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" Jul 13 18:58:52.976909 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.976873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.976929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.976957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977062 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.977041491 +0000 UTC m=+38.005599831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : configmap references non-existent config key: service-ca.crt Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977064 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977080 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977102 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.977093441 +0000 UTC m=+38.005651756 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : secret "router-metrics-certs-default" not found Jul 13 18:58:52.977109 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977102 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb8548b8f-9swm6: secret "image-registry-tls" not found Jul 13 18:58:52.977448 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:52.977127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:52.977448 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977154 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls podName:73b27864-68b8-4b8f-84b2-47dc93109879 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.977137607 +0000 UTC m=+38.005695938 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls") pod "image-registry-6fb8548b8f-9swm6" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879") : secret "image-registry-tls" not found Jul 13 18:58:52.977448 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977215 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:52.977448 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:52.977279 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls podName:e8dac37f-f511-4f56-9746-827c3213809c nodeName:}" failed. No retries permitted until 2026-07-13 18:58:53.977260697 +0000 UTC m=+38.005819013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tnch" (UID: "e8dac37f-f511-4f56-9746-827c3213809c") : secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:53.078629 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.078587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:53.078817 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.078734 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:53.078817 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.078777 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 18:58:53.078945 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.078838 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 18:58:53.078945 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.078856 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls podName:84708eb3-266e-43f3-904a-8c038080bb47 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:54.078835116 +0000 UTC m=+38.107393450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls") pod "cluster-samples-operator-65db858894-hdzft" (UID: "84708eb3-266e-43f3-904a-8c038080bb47") : secret "samples-operator-tls" not found Jul 13 18:58:53.078945 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.078892 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls podName:54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d nodeName:}" failed. No retries permitted until 2026-07-13 18:58:54.07888212 +0000 UTC m=+38.107440438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls") pod "dns-default-vf6p7" (UID: "54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d") : secret "dns-default-metrics-tls" not found Jul 13 18:58:53.079213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.078931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:53.079213 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.079074 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 18:58:53.079213 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.079131 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert podName:ac419028-3515-491f-9b60-e72e2292fea6 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:54.079113976 +0000 UTC m=+38.107672309 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8v42n" (UID: "ac419028-3515-491f-9b60-e72e2292fea6") : secret "networking-console-plugin-cert" not found Jul 13 18:58:53.548142 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.542602 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:58:53.548142 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.546411 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 18:58:53.644135 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.643706 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bk62w"] Jul 13 18:58:53.653658 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:53.653584 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23b75ff0_17af_4e4f_be25_be61ce96cfab.slice/crio-24836d9066779d6e2d0b8fa60e8449a59838eb41fce798ab52ccdfe00a735565 WatchSource:0}: Error finding container 24836d9066779d6e2d0b8fa60e8449a59838eb41fce798ab52ccdfe00a735565: Status 404 returned error can't find the container with id 24836d9066779d6e2d0b8fa60e8449a59838eb41fce798ab52ccdfe00a735565 Jul 13 18:58:53.656016 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.655934 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-kh496"] Jul 13 18:58:53.679680 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.679618 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk"] Jul 13 18:58:53.683368 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:53.683326 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32a6104_ba31_44e4_adb2_07496b9fe7d2.slice/crio-ef03ab7ca5b3787de845ac077e6e65a8d6343874d3039e54820923215351e98d WatchSource:0}: Error finding container ef03ab7ca5b3787de845ac077e6e65a8d6343874d3039e54820923215351e98d: Status 404 returned error can't find the container with id ef03ab7ca5b3787de845ac077e6e65a8d6343874d3039e54820923215351e98d Jul 13 18:58:53.687221 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.687152 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-44q2g"] Jul 13 18:58:53.690927 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.690835 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8"] Jul 13 18:58:53.691015 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:53.690994 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc33eccb_8c33_46d3_8c49_87ce85e050af.slice/crio-60f12f591a4d5cea2ccd0f4c20b45dc38e4ef12fb4b21ed3c4d821127582c62d WatchSource:0}: Error finding container 60f12f591a4d5cea2ccd0f4c20b45dc38e4ef12fb4b21ed3c4d821127582c62d: Status 404 returned error can't find the container with id 60f12f591a4d5cea2ccd0f4c20b45dc38e4ef12fb4b21ed3c4d821127582c62d Jul 13 18:58:53.694277 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:53.694256 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861e258d_9123_4ad0_8e74_d3b682a1746f.slice/crio-2e84d3e823eca21966b80a023146ac527fcdd39627eace8a6d753664fcedded1 WatchSource:0}: Error finding container 2e84d3e823eca21966b80a023146ac527fcdd39627eace8a6d753664fcedded1: Status 404 returned error can't find the container with id 2e84d3e823eca21966b80a023146ac527fcdd39627eace8a6d753664fcedded1 Jul 13 18:58:53.700156 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.699898 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59"] Jul 13 18:58:53.702822 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.702794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h"] Jul 13 18:58:53.704119 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:53.704094 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeaea0df7_d85f_48b1_b100_dacf528c63b2.slice/crio-3dbd0301225f680f43ef207e11ae1607060ad2c1e7aa3eb5e597eb1f15100380 WatchSource:0}: Error finding container 3dbd0301225f680f43ef207e11ae1607060ad2c1e7aa3eb5e597eb1f15100380: Status 404 returned error can't find the container with id 3dbd0301225f680f43ef207e11ae1607060ad2c1e7aa3eb5e597eb1f15100380 Jul 13 18:58:53.705863 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:58:53.705801 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67985222_8a17_45f7_b42c_fa011cdd7fd1.slice/crio-bd0e27cee91ef69a9001001c285371210744abe0f89f8b04b75638684db761a9 WatchSource:0}: Error finding container bd0e27cee91ef69a9001001c285371210744abe0f89f8b04b75638684db761a9: Status 404 returned error can't find the container with id bd0e27cee91ef69a9001001c285371210744abe0f89f8b04b75638684db761a9 Jul 13 18:58:53.759347 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.759307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" event={"ID":"67985222-8a17-45f7-b42c-fa011cdd7fd1","Type":"ContainerStarted","Data":"bd0e27cee91ef69a9001001c285371210744abe0f89f8b04b75638684db761a9"} Jul 13 18:58:53.760533 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.760507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" event={"ID":"fc33eccb-8c33-46d3-8c49-87ce85e050af","Type":"ContainerStarted","Data":"60f12f591a4d5cea2ccd0f4c20b45dc38e4ef12fb4b21ed3c4d821127582c62d"} Jul 13 18:58:53.761730 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.761696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bk62w" event={"ID":"23b75ff0-17af-4e4f-be25-be61ce96cfab","Type":"ContainerStarted","Data":"24836d9066779d6e2d0b8fa60e8449a59838eb41fce798ab52ccdfe00a735565"} Jul 13 18:58:53.762851 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.762827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" event={"ID":"b32a6104-ba31-44e4-adb2-07496b9fe7d2","Type":"ContainerStarted","Data":"ef03ab7ca5b3787de845ac077e6e65a8d6343874d3039e54820923215351e98d"} Jul 13 18:58:53.765822 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.765788 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerStarted","Data":"9f7af01741152d338bfa0377bbd0d25a6e38e8473207394f89e6c166cb0db485"} Jul 13 18:58:53.766953 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.766932 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" event={"ID":"861e258d-9123-4ad0-8e74-d3b682a1746f","Type":"ContainerStarted","Data":"2e84d3e823eca21966b80a023146ac527fcdd39627eace8a6d753664fcedded1"} Jul 13 18:58:53.768049 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.768031 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" event={"ID":"eaea0df7-d85f-48b1-b100-dacf528c63b2","Type":"ContainerStarted","Data":"3dbd0301225f680f43ef207e11ae1607060ad2c1e7aa3eb5e597eb1f15100380"} Jul 13 18:58:53.769016 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.768993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kh496" event={"ID":"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e","Type":"ContainerStarted","Data":"90f495a82a27a2b760f7a6e9f220103193ba4b41818cc86131b89f541df1e301"} Jul 13 18:58:53.992470 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.992378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:53.992470 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.992428 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.992476 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992516 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992535 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb8548b8f-9swm6: secret "image-registry-tls" not found Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.992569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992583 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls podName:73b27864-68b8-4b8f-84b2-47dc93109879 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:55.992567793 +0000 UTC m=+40.021126107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls") pod "image-registry-6fb8548b8f-9swm6" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879") : secret "image-registry-tls" not found Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992596 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992655 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:53.992704 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992689 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:55.992638866 +0000 UTC m=+40.021197185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : configmap references non-existent config key: service-ca.crt Jul 13 18:58:53.992962 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.992719 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:55.99271224 +0000 UTC m=+40.021270555 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : secret "router-metrics-certs-default" not found Jul 13 18:58:53.993196 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:53.993181 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls podName:e8dac37f-f511-4f56-9746-827c3213809c nodeName:}" failed. No retries permitted until 2026-07-13 18:58:55.993164655 +0000 UTC m=+40.021722988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tnch" (UID: "e8dac37f-f511-4f56-9746-827c3213809c") : secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:53.999786 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:53.999759 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:54.071598 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.071554 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:54.093487 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.093450 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:54.093487 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.093491 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:54.093717 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.093574 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:54.093717 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:54.093617 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 18:58:54.093717 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:54.093700 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 18:58:54.093834 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:54.093745 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 18:58:54.093834 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:54.093714 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert podName:ac419028-3515-491f-9b60-e72e2292fea6 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:56.093693003 +0000 UTC m=+40.122251334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8v42n" (UID: "ac419028-3515-491f-9b60-e72e2292fea6") : secret "networking-console-plugin-cert" not found Jul 13 18:58:54.093834 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:54.093802 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls podName:84708eb3-266e-43f3-904a-8c038080bb47 nodeName:}" failed. No retries permitted until 2026-07-13 18:58:56.093785803 +0000 UTC m=+40.122344122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls") pod "cluster-samples-operator-65db858894-hdzft" (UID: "84708eb3-266e-43f3-904a-8c038080bb47") : secret "samples-operator-tls" not found Jul 13 18:58:54.093834 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:54.093822 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls podName:54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d nodeName:}" failed. No retries permitted until 2026-07-13 18:58:56.093810383 +0000 UTC m=+40.122368699 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls") pod "dns-default-vf6p7" (UID: "54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d") : secret "dns-default-metrics-tls" not found Jul 13 18:58:54.668009 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.667919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:54.779193 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.778110 2580 generic.go:356] "Generic (PLEG): container finished" podID="4061c19f-9533-4feb-bcf2-3f42c037565e" containerID="9f7af01741152d338bfa0377bbd0d25a6e38e8473207394f89e6c166cb0db485" exitCode=0 Jul 13 18:58:54.779193 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:54.778207 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerDied","Data":"9f7af01741152d338bfa0377bbd0d25a6e38e8473207394f89e6c166cb0db485"} Jul 13 18:58:55.788715 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:55.788408 2580 generic.go:356] "Generic (PLEG): container finished" podID="4061c19f-9533-4feb-bcf2-3f42c037565e" containerID="be0462c4eff8d2c5c777aab5d41766e534eb9a4498cbdfcb33196acf1ffac2d9" exitCode=0 Jul 13 18:58:55.789111 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:55.788743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerDied","Data":"be0462c4eff8d2c5c777aab5d41766e534eb9a4498cbdfcb33196acf1ffac2d9"} Jul 13 18:58:56.004538 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.004491 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:56.016247 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.016202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:56.016453 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.016274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:58:56.016453 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.016304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:58:56.016453 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.016372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:58:56.016629 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.016616 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.016703 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls podName:e8dac37f-f511-4f56-9746-827c3213809c nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.016665615 +0000 UTC m=+44.045223945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tnch" (UID: "e8dac37f-f511-4f56-9746-827c3213809c") : secret "cluster-monitoring-operator-tls" not found Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.017141 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.017124477 +0000 UTC m=+44.045682799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : configmap references non-existent config key: service-ca.crt Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.017226 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.017239 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb8548b8f-9swm6: secret "image-registry-tls" not found Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.017275 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls podName:73b27864-68b8-4b8f-84b2-47dc93109879 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.017262109 +0000 UTC m=+44.045820433 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls") pod "image-registry-6fb8548b8f-9swm6" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879") : secret "image-registry-tls" not found Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.017339 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 18:58:56.017402 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.017373 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.017360535 +0000 UTC m=+44.045918855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : secret "router-metrics-certs-default" not found Jul 13 18:58:56.102267 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.102229 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" limit=1024 Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.117329 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.117381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:58:56.117480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.117520 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.117599 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert podName:ac419028-3515-491f-9b60-e72e2292fea6 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.117577789 +0000 UTC m=+44.146136118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8v42n" (UID: "ac419028-3515-491f-9b60-e72e2292fea6") : secret "networking-console-plugin-cert" not found Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.117637 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 18:58:56.117733 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.117642 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 18:58:56.118136 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.117707 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls podName:54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.117691529 +0000 UTC m=+44.146249844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls") pod "dns-default-vf6p7" (UID: "54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d") : secret "dns-default-metrics-tls" not found Jul 13 18:58:56.118136 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:58:56.117822 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls podName:84708eb3-266e-43f3-904a-8c038080bb47 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:00.117805579 +0000 UTC m=+44.146363897 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls") pod "cluster-samples-operator-65db858894-hdzft" (UID: "84708eb3-266e-43f3-904a-8c038080bb47") : secret "samples-operator-tls" not found Jul 13 18:59:00.057593 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.057548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.057610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.057640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.057721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.057772 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.057744604 +0000 UTC m=+52.086302921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : configmap references non-existent config key: service-ca.crt Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.057797 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.057813 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.057814 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb8548b8f-9swm6: secret "image-registry-tls" not found Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.057870 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.057877 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls podName:e8dac37f-f511-4f56-9746-827c3213809c nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.057862941 +0000 UTC m=+52.086421260 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tnch" (UID: "e8dac37f-f511-4f56-9746-827c3213809c") : secret "cluster-monitoring-operator-tls" not found Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.058011 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls podName:73b27864-68b8-4b8f-84b2-47dc93109879 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.057985533 +0000 UTC m=+52.086543852 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls") pod "image-registry-6fb8548b8f-9swm6" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879") : secret "image-registry-tls" not found Jul 13 18:59:00.058055 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.058039 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.058029022 +0000 UTC m=+52.086587337 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : secret "router-metrics-certs-default" not found Jul 13 18:59:00.158729 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.158684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:59:00.158939 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.158742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:59:00.158939 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:00.158821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:00.158939 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.158836 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 18:59:00.158939 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.158870 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 13 18:59:00.158939 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.158911 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert podName:ac419028-3515-491f-9b60-e72e2292fea6 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.158890353 +0000 UTC m=+52.187448693 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8v42n" (UID: "ac419028-3515-491f-9b60-e72e2292fea6") : secret "networking-console-plugin-cert" not found Jul 13 18:59:00.158939 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.158930 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls podName:84708eb3-266e-43f3-904a-8c038080bb47 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.158920586 +0000 UTC m=+52.187478900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls") pod "cluster-samples-operator-65db858894-hdzft" (UID: "84708eb3-266e-43f3-904a-8c038080bb47") : secret "samples-operator-tls" not found Jul 13 18:59:00.159202 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.158984 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 18:59:00.159202 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:00.159034 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls podName:54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d nodeName:}" failed. No retries permitted until 2026-07-13 18:59:08.159020301 +0000 UTC m=+52.187578615 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls") pod "dns-default-vf6p7" (UID: "54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d") : secret "dns-default-metrics-tls" not found Jul 13 18:59:01.809838 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.809459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" event={"ID":"67985222-8a17-45f7-b42c-fa011cdd7fd1","Type":"ContainerStarted","Data":"46b93906891a72b7e541745b81d95e3e0157d70b21fafaebe886ae8e741e3770"} Jul 13 18:59:01.811473 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.811444 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" event={"ID":"fc33eccb-8c33-46d3-8c49-87ce85e050af","Type":"ContainerStarted","Data":"c959f971538fc80c3cbb41ebca11d623ef5200f4f31024f2abb36c203c300f03"} Jul 13 18:59:01.813251 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.813158 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bk62w" event={"ID":"23b75ff0-17af-4e4f-be25-be61ce96cfab","Type":"ContainerStarted","Data":"cfed1d2876bb8a1d6c7a9a126ceeaaf583f40b647bdd95819610ddace5d9acf6"} Jul 13 18:59:01.815205 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.814842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" event={"ID":"b32a6104-ba31-44e4-adb2-07496b9fe7d2","Type":"ContainerStarted","Data":"f0dbd8e1f57e17b43256cd825d86ece1f5cc1b66a3050fcb5df7a39b2e67b4e7"} Jul 13 18:59:01.818931 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.818903 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8l58j" event={"ID":"4061c19f-9533-4feb-bcf2-3f42c037565e","Type":"ContainerStarted","Data":"2c6a3e9b7115e62ce3476a73b5114ea34a780c2d7739b4090057d3f0b86dd94b"} Jul 13 18:59:01.820884 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.820705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" event={"ID":"861e258d-9123-4ad0-8e74-d3b682a1746f","Type":"ContainerStarted","Data":"fe35e2dc829efb612d8bcceadd2391812a986d9df84513690f8f4a8f26b9b991"} Jul 13 18:59:01.822305 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.822280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" event={"ID":"eaea0df7-d85f-48b1-b100-dacf528c63b2","Type":"ContainerStarted","Data":"9c0cded27ad0ab3b715a270fc9a375b32e1fd1ddc82e5c535dbd0ac26c35c327"} Jul 13 18:59:01.824095 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.824073 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/0.log" Jul 13 18:59:01.824213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.824116 2580 generic.go:356] "Generic (PLEG): container finished" podID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" containerID="e592bcbb8eb7294c306e25c69b9da344cd467a22afdab839ce882f5c968b99cb" exitCode=255 Jul 13 18:59:01.824213 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.824166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kh496" event={"ID":"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e","Type":"ContainerDied","Data":"e592bcbb8eb7294c306e25c69b9da344cd467a22afdab839ce882f5c968b99cb"} Jul 13 18:59:01.824370 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.824356 2580 scope.go:117] "RemoveContainer" containerID="e592bcbb8eb7294c306e25c69b9da344cd467a22afdab839ce882f5c968b99cb" Jul 13 18:59:01.827686 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.827579 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" podStartSLOduration=18.299444622 podStartE2EDuration="25.827564049s" totalImagesPullingTime="7.528119427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.826505982 +0000 UTC m=+45.855064319" watchObservedRunningTime="2026-07-13 18:59:01.827564049 +0000 UTC m=+45.856122384" Jul 13 18:59:01.842906 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.842397 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-zh6fk" podStartSLOduration=18.138615511 podStartE2EDuration="25.842376449s" totalImagesPullingTime="7.703760938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.841550153 +0000 UTC m=+45.870108487" watchObservedRunningTime="2026-07-13 18:59:01.842376449 +0000 UTC m=+45.870934787" Jul 13 18:59:01.860877 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.860131 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bk62w" podStartSLOduration=2.130843719 podStartE2EDuration="9.860111837s" totalImagesPullingTime="7.729268118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.85921529 +0000 UTC m=+45.887773627" watchObservedRunningTime="2026-07-13 18:59:01.860111837 +0000 UTC m=+45.888670175" Jul 13 18:59:01.897378 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.897300 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" podStartSLOduration=18.201670249 podStartE2EDuration="25.897276261s" totalImagesPullingTime="7.695606012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.876452668 +0000 UTC m=+45.905011006" watchObservedRunningTime="2026-07-13 18:59:01.897276261 +0000 UTC m=+45.925834577" Jul 13 18:59:01.922157 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.920584 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" podStartSLOduration=18.238156292 podStartE2EDuration="25.920561946s" totalImagesPullingTime="7.682405654s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.899062194 +0000 UTC m=+45.927620536" watchObservedRunningTime="2026-07-13 18:59:01.920561946 +0000 UTC m=+45.949120286" Jul 13 18:59:01.945447 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.944715 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-7vrr8" podStartSLOduration=18.252038248 podStartE2EDuration="25.944691273s" totalImagesPullingTime="7.692653025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.922225808 +0000 UTC m=+45.950784145" watchObservedRunningTime="2026-07-13 18:59:01.944691273 +0000 UTC m=+45.973249607" Jul 13 18:59:01.970904 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:01.970840 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8l58j" podStartSLOduration=18.9832139 podStartE2EDuration="45.970821949s" totalImagesPullingTime="26.987608049s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:01.969983376 +0000 UTC m=+45.998541737" watchObservedRunningTime="2026-07-13 18:59:01.970821949 +0000 UTC m=+45.999380284" Jul 13 18:59:02.663178 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.663138 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:02.663360 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.663199 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:02.833359 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.833328 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/1.log" Jul 13 18:59:02.833832 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.833802 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/0.log" Jul 13 18:59:02.833900 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.833845 2580 generic.go:356] "Generic (PLEG): container finished" podID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" containerID="82aea8ba13628b353098ab4c2444b39aef10325e6b3c4fc46964f06d9b5a71ed" exitCode=255 Jul 13 18:59:02.834053 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.834013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kh496" event={"ID":"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e","Type":"ContainerDied","Data":"82aea8ba13628b353098ab4c2444b39aef10325e6b3c4fc46964f06d9b5a71ed"} Jul 13 18:59:02.834189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.834121 2580 scope.go:117] "RemoveContainer" containerID="e592bcbb8eb7294c306e25c69b9da344cd467a22afdab839ce882f5c968b99cb" Jul 13 18:59:02.834189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.834168 2580 scope.go:117] "RemoveContainer" containerID="82aea8ba13628b353098ab4c2444b39aef10325e6b3c4fc46964f06d9b5a71ed" Jul 13 18:59:02.834920 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:02.834362 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kh496_openshift-console-operator(5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e)\"" pod="openshift-console-operator/console-operator-7866497494-kh496" podUID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" Jul 13 18:59:02.944559 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.944472 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6"] Jul 13 18:59:02.974186 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.974151 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6"] Jul 13 18:59:02.974376 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.974272 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" Jul 13 18:59:02.977062 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.977020 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 13 18:59:02.977062 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.977037 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-nkv4d\"" Jul 13 18:59:02.977276 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:02.977066 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 13 18:59:03.090780 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.090737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgvnd\" (UniqueName: \"kubernetes.io/projected/e8bc330b-c3a9-40c1-a071-4154d117a9da-kube-api-access-bgvnd\") pod \"migrator-7f59b54f6c-q8cj6\" (UID: \"e8bc330b-c3a9-40c1-a071-4154d117a9da\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" Jul 13 18:59:03.191952 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.191909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgvnd\" (UniqueName: \"kubernetes.io/projected/e8bc330b-c3a9-40c1-a071-4154d117a9da-kube-api-access-bgvnd\") pod \"migrator-7f59b54f6c-q8cj6\" (UID: \"e8bc330b-c3a9-40c1-a071-4154d117a9da\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" Jul 13 18:59:03.208312 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.208222 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgvnd\" (UniqueName: \"kubernetes.io/projected/e8bc330b-c3a9-40c1-a071-4154d117a9da-kube-api-access-bgvnd\") pod \"migrator-7f59b54f6c-q8cj6\" (UID: \"e8bc330b-c3a9-40c1-a071-4154d117a9da\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" Jul 13 18:59:03.284831 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.284793 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" Jul 13 18:59:03.341718 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.341659 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x7mw8"] Jul 13 18:59:03.374376 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.374303 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.375350 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.375327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x7mw8"] Jul 13 18:59:03.377510 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.377277 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 18:59:03.377641 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.377549 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 18:59:03.377916 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.377750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d2l29\"" Jul 13 18:59:03.463568 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.463480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6"] Jul 13 18:59:03.466472 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:03.466430 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8bc330b_c3a9_40c1_a071_4154d117a9da.slice/crio-198ebedc5aa50d0e62e5438475630cd73c0c44e27178bbf919a23b08239fef0b WatchSource:0}: Error finding container 198ebedc5aa50d0e62e5438475630cd73c0c44e27178bbf919a23b08239fef0b: Status 404 returned error can't find the container with id 198ebedc5aa50d0e62e5438475630cd73c0c44e27178bbf919a23b08239fef0b Jul 13 18:59:03.495062 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.495023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b40a891-5b0a-4f46-b993-0b94d4a5a142-crio-socket\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.495062 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.495079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b40a891-5b0a-4f46-b993-0b94d4a5a142-data-volume\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.495256 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.495126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.495324 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.495284 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl24t\" (UniqueName: \"kubernetes.io/projected/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-api-access-rl24t\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.495324 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.495317 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.595819 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.595780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.596038 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.595946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl24t\" (UniqueName: \"kubernetes.io/projected/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-api-access-rl24t\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.596038 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.595974 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.596038 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:03.595979 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 18:59:03.596038 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.596015 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b40a891-5b0a-4f46-b993-0b94d4a5a142-crio-socket\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.596038 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.596037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b40a891-5b0a-4f46-b993-0b94d4a5a142-data-volume\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.596367 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:03.596059 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:04.09603684 +0000 UTC m=+48.124595160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : configmap "kube-rbac-proxy" not found Jul 13 18:59:03.596367 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:03.596126 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 18:59:03.596367 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:03.596193 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:04.096165962 +0000 UTC m=+48.124724277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : secret "insights-runtime-extractor-tls" not found Jul 13 18:59:03.596367 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.596342 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4b40a891-5b0a-4f46-b993-0b94d4a5a142-crio-socket\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.596561 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.596379 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b40a891-5b0a-4f46-b993-0b94d4a5a142-data-volume\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.605292 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.605258 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl24t\" (UniqueName: \"kubernetes.io/projected/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-api-access-rl24t\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:03.839949 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.839915 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/1.log" Jul 13 18:59:03.840407 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.840327 2580 scope.go:117] "RemoveContainer" containerID="82aea8ba13628b353098ab4c2444b39aef10325e6b3c4fc46964f06d9b5a71ed" Jul 13 18:59:03.840548 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:03.840524 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-kh496_openshift-console-operator(5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e)\"" pod="openshift-console-operator/console-operator-7866497494-kh496" podUID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" Jul 13 18:59:03.841219 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:03.841200 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" event={"ID":"e8bc330b-c3a9-40c1-a071-4154d117a9da","Type":"ContainerStarted","Data":"198ebedc5aa50d0e62e5438475630cd73c0c44e27178bbf919a23b08239fef0b"} Jul 13 18:59:04.101834 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.101736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:04.101834 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.101831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:04.102022 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:04.101877 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 18:59:04.102022 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:04.101914 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 18:59:04.102022 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:04.101939 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:05.101922091 +0000 UTC m=+49.130480405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : secret "insights-runtime-extractor-tls" not found Jul 13 18:59:04.102022 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:04.101956 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:05.101944895 +0000 UTC m=+49.130503209 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : configmap "kube-rbac-proxy" not found Jul 13 18:59:04.821684 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.821629 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g2rbm"] Jul 13 18:59:04.842039 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.842010 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g2rbm"] Jul 13 18:59:04.842497 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.842077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:04.844737 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.844711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 13 18:59:04.844737 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.844730 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 13 18:59:04.845614 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.845595 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 13 18:59:04.845737 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.845655 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gwqdt\"" Jul 13 18:59:04.845737 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.845717 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 13 18:59:04.909892 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.909853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-signing-key\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:04.910156 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.910126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-signing-cabundle\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:04.910218 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:04.910186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwvh9\" (UniqueName: \"kubernetes.io/projected/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-kube-api-access-zwvh9\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.011208 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.011168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-signing-cabundle\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.011414 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.011230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwvh9\" (UniqueName: \"kubernetes.io/projected/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-kube-api-access-zwvh9\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.011414 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.011304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-signing-key\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.011991 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.011969 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-signing-cabundle\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.013906 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.013885 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-signing-key\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.020053 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.020025 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwvh9\" (UniqueName: \"kubernetes.io/projected/f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e-kube-api-access-zwvh9\") pod \"service-ca-79f68b9f4f-g2rbm\" (UID: \"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.112484 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.112292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:05.112484 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.112389 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:05.112484 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:05.112436 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 18:59:05.112832 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:05.112519 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:07.112494975 +0000 UTC m=+51.141053300 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : secret "insights-runtime-extractor-tls" not found Jul 13 18:59:05.112832 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:05.112532 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 18:59:05.112832 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:05.112574 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:07.112562753 +0000 UTC m=+51.141121068 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : configmap "kube-rbac-proxy" not found Jul 13 18:59:05.134824 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.134774 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fmnt7_0b4404b1-b307-4b37-b215-b250669247b7/dns-node-resolver/0.log" Jul 13 18:59:05.157024 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.156982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" Jul 13 18:59:05.343142 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.343120 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g2rbm"] Jul 13 18:59:05.345440 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:05.345413 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e7e8ee_6f6f_4afb_bb1e_afb4dc54ce6e.slice/crio-5667d588518b41ac860394af98c3538cdbb0c1baa5074fc9c7c4cd4c48eb9fea WatchSource:0}: Error finding container 5667d588518b41ac860394af98c3538cdbb0c1baa5074fc9c7c4cd4c48eb9fea: Status 404 returned error can't find the container with id 5667d588518b41ac860394af98c3538cdbb0c1baa5074fc9c7c4cd4c48eb9fea Jul 13 18:59:05.853535 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.853497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" event={"ID":"e8bc330b-c3a9-40c1-a071-4154d117a9da","Type":"ContainerStarted","Data":"fedb19d09c37cb3bbf94c029d102fe940dae56b34722f7138d8933cccc2c99b2"} Jul 13 18:59:05.855006 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.854979 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" event={"ID":"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e","Type":"ContainerStarted","Data":"5a8c0f0f3a1d9b70854edaf782df5193c1221a31524801d956debf0410c90084"} Jul 13 18:59:05.855281 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.855025 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" event={"ID":"f3e7e8ee-6f6f-4afb-bb1e-afb4dc54ce6e","Type":"ContainerStarted","Data":"5667d588518b41ac860394af98c3538cdbb0c1baa5074fc9c7c4cd4c48eb9fea"} Jul 13 18:59:05.894810 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:05.894749 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-g2rbm" podStartSLOduration=1.894724826 podStartE2EDuration="1.894724826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:05.893086428 +0000 UTC m=+49.921644768" watchObservedRunningTime="2026-07-13 18:59:05.894724826 +0000 UTC m=+49.923283163" Jul 13 18:59:06.335943 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:06.335914 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pkh94_fa191c78-a11e-4519-a32b-c036c00c12cf/node-ca/0.log" Jul 13 18:59:06.860487 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:06.860453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" event={"ID":"e8bc330b-c3a9-40c1-a071-4154d117a9da","Type":"ContainerStarted","Data":"576f22fd2a79b230219b52ca1b3786f513e1798fc334d9025aadbc3d575ffbb4"} Jul 13 18:59:06.880662 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:06.880593 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-q8cj6" podStartSLOduration=2.699127716 podStartE2EDuration="4.880568683s" totalImagesPullingTime="2.181440967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:59:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:06.8795558 +0000 UTC m=+50.908114148" watchObservedRunningTime="2026-07-13 18:59:06.880568683 +0000 UTC m=+50.909127389" Jul 13 18:59:06.936532 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:06.936495 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bk62w_23b75ff0-17af-4e4f-be25-be61ce96cfab/serve-healthcheck-canary/0.log" Jul 13 18:59:07.132355 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:07.132249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:07.132538 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:07.132359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:07.132538 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:07.132449 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 18:59:07.132538 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:07.132493 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 18:59:07.132538 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:07.132535 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:11.132514277 +0000 UTC m=+55.161072607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : secret "insights-runtime-extractor-tls" not found Jul 13 18:59:07.132719 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:07.132558 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:11.132548011 +0000 UTC m=+55.161106330 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : configmap "kube-rbac-proxy" not found Jul 13 18:59:07.141237 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:07.141208 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-q8cj6_e8bc330b-c3a9-40c1-a071-4154d117a9da/migrator/0.log" Jul 13 18:59:07.345189 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:07.345151 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-q8cj6_e8bc330b-c3a9-40c1-a071-4154d117a9da/graceful-termination/0.log" Jul 13 18:59:07.545260 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:07.545170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-ngr59_eaea0df7-d85f-48b1-b100-dacf528c63b2/kube-storage-version-migrator-operator/0.log" Jul 13 18:59:08.143312 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.143269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.143324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.143381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143454 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143470 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6fb8548b8f-9swm6: secret "image-registry-tls" not found Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143536 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls podName:73b27864-68b8-4b8f-84b2-47dc93109879 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:24.143510194 +0000 UTC m=+68.172068512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls") pod "image-registry-6fb8548b8f-9swm6" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879") : secret "image-registry-tls" not found Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.143466 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143585 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143614 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143591 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:24.143570917 +0000 UTC m=+68.172129239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : configmap references non-existent config key: service-ca.crt Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143667 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls podName:e8dac37f-f511-4f56-9746-827c3213809c nodeName:}" failed. No retries permitted until 2026-07-13 18:59:24.143655082 +0000 UTC m=+68.172213396 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-6tnch" (UID: "e8dac37f-f511-4f56-9746-827c3213809c") : secret "cluster-monitoring-operator-tls" not found Jul 13 18:59:08.143772 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.143697 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs podName:a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:24.143688887 +0000 UTC m=+68.172247203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs") pod "router-default-57897d6b58-6nnn2" (UID: "a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7") : secret "router-metrics-certs-default" not found Jul 13 18:59:08.244518 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.244467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:08.244722 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.244599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:59:08.244722 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.244620 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:59:08.244722 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.244624 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 18:59:08.244850 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.244729 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls podName:54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d nodeName:}" failed. No retries permitted until 2026-07-13 18:59:24.244711873 +0000 UTC m=+68.273270188 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls") pod "dns-default-vf6p7" (UID: "54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d") : secret "dns-default-metrics-tls" not found Jul 13 18:59:08.244850 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.244772 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 13 18:59:08.244850 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:08.244837 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert podName:ac419028-3515-491f-9b60-e72e2292fea6 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:24.244820263 +0000 UTC m=+68.273378583 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-8v42n" (UID: "ac419028-3515-491f-9b60-e72e2292fea6") : secret "networking-console-plugin-cert" not found Jul 13 18:59:08.247021 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.246996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/84708eb3-266e-43f3-904a-8c038080bb47-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-hdzft\" (UID: \"84708eb3-266e-43f3-904a-8c038080bb47\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:59:08.252549 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.252522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" Jul 13 18:59:08.401490 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.401402 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft"] Jul 13 18:59:08.868850 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:08.868804 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" event={"ID":"84708eb3-266e-43f3-904a-8c038080bb47","Type":"ContainerStarted","Data":"05c4255382c3b97eca151db905529eb61c81f219d9d2364495b782cb44bf674c"} Jul 13 18:59:11.174552 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:11.174451 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:11.175014 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:11.174594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:11.175014 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:11.174773 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 18:59:11.175014 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:11.174858 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls podName:4b40a891-5b0a-4f46-b993-0b94d4a5a142 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:19.174836316 +0000 UTC m=+63.203394631 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x7mw8" (UID: "4b40a891-5b0a-4f46-b993-0b94d4a5a142") : secret "insights-runtime-extractor-tls" not found Jul 13 18:59:11.175174 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:11.175134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4b40a891-5b0a-4f46-b993-0b94d4a5a142-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:11.881730 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:11.881691 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" event={"ID":"84708eb3-266e-43f3-904a-8c038080bb47","Type":"ContainerStarted","Data":"680b536f7db196e0d124a5ce2008b4eae28e14d2fba7cd656b73f0cd0163118e"} Jul 13 18:59:11.881912 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:11.881755 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" event={"ID":"84708eb3-266e-43f3-904a-8c038080bb47","Type":"ContainerStarted","Data":"a92685f4b7d71ce6d2369860d057a83a3341bfa07a3000f2b5af23a1cbd5b3e0"} Jul 13 18:59:11.903564 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:11.903507 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-hdzft" podStartSLOduration=33.513018111 podStartE2EDuration="35.903491548s" totalImagesPullingTime="2.390473437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:11.901295202 +0000 UTC m=+55.929853538" watchObservedRunningTime="2026-07-13 18:59:11.903491548 +0000 UTC m=+55.932049862" Jul 13 18:59:12.662640 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.662595 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:12.663251 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.662666 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:12.663251 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.663108 2580 scope.go:117] "RemoveContainer" containerID="82aea8ba13628b353098ab4c2444b39aef10325e6b3c4fc46964f06d9b5a71ed" Jul 13 18:59:12.886138 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.886108 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 18:59:12.886524 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.886507 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/1.log" Jul 13 18:59:12.886576 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.886542 2580 generic.go:356] "Generic (PLEG): container finished" podID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" containerID="18718bc6336c08fd75f661c1598475d51cecbb724d28f7979a5c4f9c727a5a45" exitCode=255 Jul 13 18:59:12.886634 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.886614 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kh496" event={"ID":"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e","Type":"ContainerDied","Data":"18718bc6336c08fd75f661c1598475d51cecbb724d28f7979a5c4f9c727a5a45"} Jul 13 18:59:12.886698 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.886688 2580 scope.go:117] "RemoveContainer" containerID="82aea8ba13628b353098ab4c2444b39aef10325e6b3c4fc46964f06d9b5a71ed" Jul 13 18:59:12.887112 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:12.887094 2580 scope.go:117] "RemoveContainer" containerID="18718bc6336c08fd75f661c1598475d51cecbb724d28f7979a5c4f9c727a5a45" Jul 13 18:59:12.887275 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:12.887253 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-kh496_openshift-console-operator(5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e)\"" pod="openshift-console-operator/console-operator-7866497494-kh496" podUID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" Jul 13 18:59:13.897989 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:13.897961 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 18:59:17.755686 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:17.755646 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nmg8g" Jul 13 18:59:19.248839 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:19.248790 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:19.251119 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:19.251097 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4b40a891-5b0a-4f46-b993-0b94d4a5a142-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x7mw8\" (UID: \"4b40a891-5b0a-4f46-b993-0b94d4a5a142\") " pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:19.294436 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:19.294405 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d2l29\"" Jul 13 18:59:19.301890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:19.301862 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x7mw8" Jul 13 18:59:19.434568 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:19.434477 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x7mw8"] Jul 13 18:59:19.437203 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:19.437177 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b40a891_5b0a_4f46_b993_0b94d4a5a142.slice/crio-6e3e203b8dbe1485359949d43ac10c57db8fc7ad69a7baf921794f391b126e48 WatchSource:0}: Error finding container 6e3e203b8dbe1485359949d43ac10c57db8fc7ad69a7baf921794f391b126e48: Status 404 returned error can't find the container with id 6e3e203b8dbe1485359949d43ac10c57db8fc7ad69a7baf921794f391b126e48 Jul 13 18:59:19.916717 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:19.916652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7mw8" event={"ID":"4b40a891-5b0a-4f46-b993-0b94d4a5a142","Type":"ContainerStarted","Data":"6e3e203b8dbe1485359949d43ac10c57db8fc7ad69a7baf921794f391b126e48"} Jul 13 18:59:22.178411 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.178365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:59:22.178889 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.178469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:59:22.181246 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.181216 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 18:59:22.181246 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.181216 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 18:59:22.191941 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.191881 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06c1ae6a-23e4-4530-8071-496c47f2f077-metrics-certs\") pod \"network-metrics-daemon-bpzhc\" (UID: \"06c1ae6a-23e4-4530-8071-496c47f2f077\") " pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:59:22.191941 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.191920 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa-original-pull-secret\") pod \"global-pull-secret-syncer-zsgck\" (UID: \"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa\") " pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:59:22.271890 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.271854 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cvxxg\"" Jul 13 18:59:22.279528 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.279494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:59:22.279735 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.279537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-bpzhc" Jul 13 18:59:22.282400 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.282363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnrn6\" (UniqueName: \"kubernetes.io/projected/3fa7146c-160b-4480-9755-33c605de0107-kube-api-access-lnrn6\") pod \"network-check-target-vk9kv\" (UID: \"3fa7146c-160b-4480-9755-33c605de0107\") " pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:59:22.391624 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.391586 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zsgck" Jul 13 18:59:22.565577 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.565503 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bqf9d\"" Jul 13 18:59:22.572839 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.572801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:59:22.662778 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.662734 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:22.662954 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.662805 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:22.663182 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:22.663167 2580 scope.go:117] "RemoveContainer" containerID="18718bc6336c08fd75f661c1598475d51cecbb724d28f7979a5c4f9c727a5a45" Jul 13 18:59:22.663386 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:22.663365 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-kh496_openshift-console-operator(5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e)\"" pod="openshift-console-operator/console-operator-7866497494-kh496" podUID="5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e" Jul 13 18:59:23.283880 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.283733 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vk9kv"] Jul 13 18:59:23.287963 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:23.287439 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fa7146c_160b_4480_9755_33c605de0107.slice/crio-ffcc2d31ced4d24d4b7e13c14acdda52fbcbed7e0200f193a03c3a191f3e0423 WatchSource:0}: Error finding container ffcc2d31ced4d24d4b7e13c14acdda52fbcbed7e0200f193a03c3a191f3e0423: Status 404 returned error can't find the container with id ffcc2d31ced4d24d4b7e13c14acdda52fbcbed7e0200f193a03c3a191f3e0423 Jul 13 18:59:23.299138 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.299033 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-bpzhc"] Jul 13 18:59:23.301964 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:23.301938 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c1ae6a_23e4_4530_8071_496c47f2f077.slice/crio-98942f38b1b1d454c5dfc26197da91b847ad5a8b52de93d0a73dfd45496c9e3d WatchSource:0}: Error finding container 98942f38b1b1d454c5dfc26197da91b847ad5a8b52de93d0a73dfd45496c9e3d: Status 404 returned error can't find the container with id 98942f38b1b1d454c5dfc26197da91b847ad5a8b52de93d0a73dfd45496c9e3d Jul 13 18:59:23.321415 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.321329 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zsgck"] Jul 13 18:59:23.323262 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:23.323204 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a86f0ea_4e77_4f6a_a5eb_9c2af94cccaa.slice/crio-f54a6b17bc1032329cf45edd1a8e0eea1952c0388c108bd8285169b7d76e7181 WatchSource:0}: Error finding container f54a6b17bc1032329cf45edd1a8e0eea1952c0388c108bd8285169b7d76e7181: Status 404 returned error can't find the container with id f54a6b17bc1032329cf45edd1a8e0eea1952c0388c108bd8285169b7d76e7181 Jul 13 18:59:23.935163 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.935125 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bpzhc" event={"ID":"06c1ae6a-23e4-4530-8071-496c47f2f077","Type":"ContainerStarted","Data":"98942f38b1b1d454c5dfc26197da91b847ad5a8b52de93d0a73dfd45496c9e3d"} Jul 13 18:59:23.937784 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.937743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vk9kv" event={"ID":"3fa7146c-160b-4480-9755-33c605de0107","Type":"ContainerStarted","Data":"fc6fcbfd5a11f093767811c36a3bb19e4d1b088bb0f574e9269f83a5e1a6dcb4"} Jul 13 18:59:23.937969 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.937816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vk9kv" event={"ID":"3fa7146c-160b-4480-9755-33c605de0107","Type":"ContainerStarted","Data":"ffcc2d31ced4d24d4b7e13c14acdda52fbcbed7e0200f193a03c3a191f3e0423"} Jul 13 18:59:23.937969 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.937869 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 18:59:23.939926 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.939835 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zsgck" event={"ID":"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa","Type":"ContainerStarted","Data":"f54a6b17bc1032329cf45edd1a8e0eea1952c0388c108bd8285169b7d76e7181"} Jul 13 18:59:23.941958 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.941931 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7mw8" event={"ID":"4b40a891-5b0a-4f46-b993-0b94d4a5a142","Type":"ContainerStarted","Data":"77418d1320a96f4791d7ef40b8777630b8f94cc842b8a19c9abdd78a94a02b07"} Jul 13 18:59:23.965970 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:23.965314 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vk9kv" podStartSLOduration=66.96528744 podStartE2EDuration="1m6.96528744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:23.961956801 +0000 UTC m=+67.990515139" watchObservedRunningTime="2026-07-13 18:59:23.96528744 +0000 UTC m=+67.993845768" Jul 13 18:59:24.201238 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.201145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:24.201238 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.201191 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:24.201238 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.201217 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:24.201563 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.201266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:59:24.201990 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.201964 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-service-ca-bundle\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:24.203945 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.203917 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"image-registry-6fb8548b8f-9swm6\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:24.204060 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.204037 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7-metrics-certs\") pod \"router-default-57897d6b58-6nnn2\" (UID: \"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7\") " pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:24.204273 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.204250 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e8dac37f-f511-4f56-9746-827c3213809c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-6tnch\" (UID: \"e8dac37f-f511-4f56-9746-827c3213809c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:59:24.302510 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.302463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:59:24.303074 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.302558 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:24.305509 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.305449 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d-metrics-tls\") pod \"dns-default-vf6p7\" (UID: \"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d\") " pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:24.308412 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.308380 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ac419028-3515-491f-9b60-e72e2292fea6-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-8v42n\" (UID: \"ac419028-3515-491f-9b60-e72e2292fea6\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:59:24.390384 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.390350 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xpzcf\"" Jul 13 18:59:24.398815 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.398714 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:24.405980 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.405745 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d9lm6\"" Jul 13 18:59:24.413223 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.413164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" Jul 13 18:59:24.438600 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.438575 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-67jdm\"" Jul 13 18:59:24.446277 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.446243 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:24.473381 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.473352 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qkrht\"" Jul 13 18:59:24.481649 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.481602 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" Jul 13 18:59:24.505431 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.505395 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wt7nz\"" Jul 13 18:59:24.513755 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:24.513723 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:25.118054 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.117804 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb8548b8f-9swm6"] Jul 13 18:59:25.123653 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:25.123611 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b27864_68b8_4b8f_84b2_47dc93109879.slice/crio-3ef8f8822d85f7ceda590b89ef4851a139f1131c632284201aae571e5fbc2006 WatchSource:0}: Error finding container 3ef8f8822d85f7ceda590b89ef4851a139f1131c632284201aae571e5fbc2006: Status 404 returned error can't find the container with id 3ef8f8822d85f7ceda590b89ef4851a139f1131c632284201aae571e5fbc2006 Jul 13 18:59:25.132146 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.131972 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch"] Jul 13 18:59:25.134912 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.134577 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-57897d6b58-6nnn2"] Jul 13 18:59:25.138698 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:25.138647 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8dac37f_f511_4f56_9746_827c3213809c.slice/crio-780c20dbead1ffeeda13def2d28377238e1e910d257ff70026258844dfa0fc40 WatchSource:0}: Error finding container 780c20dbead1ffeeda13def2d28377238e1e910d257ff70026258844dfa0fc40: Status 404 returned error can't find the container with id 780c20dbead1ffeeda13def2d28377238e1e910d257ff70026258844dfa0fc40 Jul 13 18:59:25.140921 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:25.140869 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda625e2f0_529d_4d3e_a59e_ccbaf6a90ce7.slice/crio-eaa02a582f1aee4fc42c7b17b424dab7601f5dd4e4fa8bb96860f093e79693ec WatchSource:0}: Error finding container eaa02a582f1aee4fc42c7b17b424dab7601f5dd4e4fa8bb96860f093e79693ec: Status 404 returned error can't find the container with id eaa02a582f1aee4fc42c7b17b424dab7601f5dd4e4fa8bb96860f093e79693ec Jul 13 18:59:25.168855 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.168769 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vf6p7"] Jul 13 18:59:25.172459 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:25.172422 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f2bedf_2189_4e12_86a1_1fdcd0e9ad8d.slice/crio-e10f2ba89d9357b72d4d6dd6089c23a5d691114bdea6a3d470c5a86e31ea36bb WatchSource:0}: Error finding container e10f2ba89d9357b72d4d6dd6089c23a5d691114bdea6a3d470c5a86e31ea36bb: Status 404 returned error can't find the container with id e10f2ba89d9357b72d4d6dd6089c23a5d691114bdea6a3d470c5a86e31ea36bb Jul 13 18:59:25.393066 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.392921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-8v42n"] Jul 13 18:59:25.396251 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:25.396206 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac419028_3515_491f_9b60_e72e2292fea6.slice/crio-42231f672bb4f71e2c762bf4613fe7f1aa98fa62b76d4a61aefcab6fe5a64a7f WatchSource:0}: Error finding container 42231f672bb4f71e2c762bf4613fe7f1aa98fa62b76d4a61aefcab6fe5a64a7f: Status 404 returned error can't find the container with id 42231f672bb4f71e2c762bf4613fe7f1aa98fa62b76d4a61aefcab6fe5a64a7f Jul 13 18:59:25.961967 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.961917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vf6p7" event={"ID":"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d","Type":"ContainerStarted","Data":"e10f2ba89d9357b72d4d6dd6089c23a5d691114bdea6a3d470c5a86e31ea36bb"} Jul 13 18:59:25.965052 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.965003 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7mw8" event={"ID":"4b40a891-5b0a-4f46-b993-0b94d4a5a142","Type":"ContainerStarted","Data":"a31c1157a3f05ca4a88dd9aaa0b82822b8c76ed501642be0b609305e5caeea00"} Jul 13 18:59:25.968177 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.967848 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bpzhc" event={"ID":"06c1ae6a-23e4-4530-8071-496c47f2f077","Type":"ContainerStarted","Data":"e72120de6edb39cff6f313b4430306ca51f982ca48049192c2d9e3a1c8ba5df4"} Jul 13 18:59:25.968177 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.967944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-bpzhc" event={"ID":"06c1ae6a-23e4-4530-8071-496c47f2f077","Type":"ContainerStarted","Data":"3249fec1037ccae3e55198bec06b90db003f1ab2ed5807046c1019af2561ea22"} Jul 13 18:59:25.970094 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.970013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57897d6b58-6nnn2" event={"ID":"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7","Type":"ContainerStarted","Data":"22c15304360029d456f108617dc60f30b2d7310739eaa346bbb82598177c1286"} Jul 13 18:59:25.970094 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.970067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-57897d6b58-6nnn2" event={"ID":"a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7","Type":"ContainerStarted","Data":"eaa02a582f1aee4fc42c7b17b424dab7601f5dd4e4fa8bb96860f093e79693ec"} Jul 13 18:59:25.972236 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.972203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" event={"ID":"73b27864-68b8-4b8f-84b2-47dc93109879","Type":"ContainerStarted","Data":"1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b"} Jul 13 18:59:25.972368 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.972256 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" event={"ID":"73b27864-68b8-4b8f-84b2-47dc93109879","Type":"ContainerStarted","Data":"3ef8f8822d85f7ceda590b89ef4851a139f1131c632284201aae571e5fbc2006"} Jul 13 18:59:25.972368 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.972304 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:25.973997 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.973967 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" event={"ID":"ac419028-3515-491f-9b60-e72e2292fea6","Type":"ContainerStarted","Data":"42231f672bb4f71e2c762bf4613fe7f1aa98fa62b76d4a61aefcab6fe5a64a7f"} Jul 13 18:59:25.975197 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.975170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" event={"ID":"e8dac37f-f511-4f56-9746-827c3213809c","Type":"ContainerStarted","Data":"780c20dbead1ffeeda13def2d28377238e1e910d257ff70026258844dfa0fc40"} Jul 13 18:59:25.988563 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:25.988295 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-bpzhc" podStartSLOduration=68.406881572 podStartE2EDuration="1m9.988270263s" totalImagesPullingTime="1.581388691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:25.986043411 +0000 UTC m=+70.014601759" watchObservedRunningTime="2026-07-13 18:59:25.988270263 +0000 UTC m=+70.016828604" Jul 13 18:59:26.008901 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:26.008839 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" podStartSLOduration=69.008815071 podStartE2EDuration="1m9.008815071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:26.007532484 +0000 UTC m=+70.036090846" watchObservedRunningTime="2026-07-13 18:59:26.008815071 +0000 UTC m=+70.037373408" Jul 13 18:59:26.027119 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:26.027055 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-57897d6b58-6nnn2" podStartSLOduration=50.027035574 podStartE2EDuration="50.027035574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:26.026545579 +0000 UTC m=+70.055103917" watchObservedRunningTime="2026-07-13 18:59:26.027035574 +0000 UTC m=+70.055593913" Jul 13 18:59:26.446685 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:26.446630 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:26.449988 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:26.449961 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:26.981377 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:26.981344 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:26.982861 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:26.982836 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-57897d6b58-6nnn2" Jul 13 18:59:32.006017 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.005929 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x7mw8" event={"ID":"4b40a891-5b0a-4f46-b993-0b94d4a5a142","Type":"ContainerStarted","Data":"83aedfdb629886a9988b929be9e45d065a697051026a0e8db54d39253b839832"} Jul 13 18:59:32.008892 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.008855 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" event={"ID":"e8dac37f-f511-4f56-9746-827c3213809c","Type":"ContainerStarted","Data":"01cf2ad3c565875df877c8e779f430f426e9814276dc12d89cf8f454bd4c5e9b"} Jul 13 18:59:32.010948 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.010918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zsgck" event={"ID":"8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa","Type":"ContainerStarted","Data":"2b9295ed26567bfe26a2b7c1fcdee9cad32ff10a17712ee9eff3fe808c9d9eb1"} Jul 13 18:59:32.012897 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.012865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vf6p7" event={"ID":"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d","Type":"ContainerStarted","Data":"a1d72ad893dddcb3ea69b37ab8a25c81b8de2859383ccc3cac033f5b8a906c31"} Jul 13 18:59:32.031257 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.030259 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x7mw8" podStartSLOduration=17.085281315 podStartE2EDuration="29.030238801s" totalImagesPullingTime="11.944957486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:59:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:32.029095661 +0000 UTC m=+76.057653995" watchObservedRunningTime="2026-07-13 18:59:32.030238801 +0000 UTC m=+76.058797138" Jul 13 18:59:32.071697 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.071626 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zsgck" podStartSLOduration=66.692308574 podStartE2EDuration="1m15.071605637s" totalImagesPullingTime="8.379297063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 18:58:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:32.070405697 +0000 UTC m=+76.098964053" watchObservedRunningTime="2026-07-13 18:59:32.071605637 +0000 UTC m=+76.100163974" Jul 13 18:59:32.072318 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:32.072283 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-6tnch" podStartSLOduration=49.510243766 podStartE2EDuration="56.072273414s" totalImagesPullingTime="6.562029648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:32.051871636 +0000 UTC m=+76.080430010" watchObservedRunningTime="2026-07-13 18:59:32.072273414 +0000 UTC m=+76.100831753" Jul 13 18:59:33.018661 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:33.018623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vf6p7" event={"ID":"54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d","Type":"ContainerStarted","Data":"2e35929c2cd3548b871500171de47d521d6095d744821a632fc957f89cb45afe"} Jul 13 18:59:33.019166 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:33.018740 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:33.020195 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:33.020169 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" event={"ID":"ac419028-3515-491f-9b60-e72e2292fea6","Type":"ContainerStarted","Data":"8744bd255fa0aa95cc78123138d9938d5e08633508b6bf371613166a74ce1eab"} Jul 13 18:59:33.057199 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:33.057149 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vf6p7" podStartSLOduration=34.527275026 podStartE2EDuration="41.057132868s" totalImagesPullingTime="6.529857842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:33.056870194 +0000 UTC m=+77.085428530" watchObservedRunningTime="2026-07-13 18:59:33.057132868 +0000 UTC m=+77.085691205" Jul 13 18:59:33.090252 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:33.090198 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-8v42n" podStartSLOduration=48.448377038 podStartE2EDuration="55.090178584s" totalImagesPullingTime="6.641801546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:33.089491731 +0000 UTC m=+77.118050097" watchObservedRunningTime="2026-07-13 18:59:33.090178584 +0000 UTC m=+77.118736920" Jul 13 18:59:34.541805 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:34.541772 2580 scope.go:117] "RemoveContainer" containerID="18718bc6336c08fd75f661c1598475d51cecbb724d28f7979a5c4f9c727a5a45" Jul 13 18:59:35.030449 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:35.030418 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 18:59:35.030651 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:35.030519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-kh496" event={"ID":"5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e","Type":"ContainerStarted","Data":"75ad963019f6a4482a46c7f0ebec651c68668ec94a5549ee1ad012022b0baa26"} Jul 13 18:59:35.030814 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:35.030795 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:35.048479 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:35.048398 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-kh496" podStartSLOduration=51.317964057 podStartE2EDuration="59.048379909s" totalImagesPullingTime="7.730415852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 18:58:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:35.047650191 +0000 UTC m=+79.076208530" watchObservedRunningTime="2026-07-13 18:59:35.048379909 +0000 UTC m=+79.076938246" Jul 13 18:59:35.266764 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:35.266721 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-kh496" Jul 13 18:59:43.027056 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:43.026938 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vf6p7" Jul 13 18:59:44.403398 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:44.403362 2580 patch_prober.go:28] interesting pod/image-registry-6fb8548b8f-9swm6 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 18:59:44.403814 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:44.403429 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" podUID="73b27864-68b8-4b8f-84b2-47dc93109879" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 18:59:45.821452 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.821410 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2sx7h"] Jul 13 18:59:45.828599 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.828566 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.831149 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.831119 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 18:59:45.831441 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.831236 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 18:59:45.831758 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.831740 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 18:59:45.831974 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.831861 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-486zq\"" Jul 13 18:59:45.832624 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.832605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 18:59:45.976261 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-sys\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976462 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-root\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976462 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-tls\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976587 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976468 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-accelerators-collector-config\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976587 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-textfile\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976587 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnk5x\" (UniqueName: \"kubernetes.io/projected/5db6382c-38a2-4cfe-a827-cae55c994448-kube-api-access-pnk5x\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976761 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db6382c-38a2-4cfe-a827-cae55c994448-metrics-client-ca\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976761 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-wtmp\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:45.976860 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:45.976756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078013 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.077978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-sys\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078204 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-root\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078204 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-tls\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078204 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078132 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-sys\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078204 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-accelerators-collector-config\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078204 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078105 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-root\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078204 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-textfile\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078502 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:46.078243 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 13 18:59:46.078502 ip-10-0-133-70 kubenswrapper[2580]: E0713 18:59:46.078302 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-tls podName:5db6382c-38a2-4cfe-a827-cae55c994448 nodeName:}" failed. No retries permitted until 2026-07-13 18:59:46.578281387 +0000 UTC m=+90.606839714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-tls") pod "node-exporter-2sx7h" (UID: "5db6382c-38a2-4cfe-a827-cae55c994448") : secret "node-exporter-tls" not found Jul 13 18:59:46.078502 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnk5x\" (UniqueName: \"kubernetes.io/projected/5db6382c-38a2-4cfe-a827-cae55c994448-kube-api-access-pnk5x\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078502 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db6382c-38a2-4cfe-a827-cae55c994448-metrics-client-ca\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078502 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-wtmp\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078757 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078540 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-textfile\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078757 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078652 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-wtmp\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.078757 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.079016 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.078989 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5db6382c-38a2-4cfe-a827-cae55c994448-metrics-client-ca\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.079339 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.079315 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-accelerators-collector-config\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.081034 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.081010 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.092780 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.092743 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnk5x\" (UniqueName: \"kubernetes.io/projected/5db6382c-38a2-4cfe-a827-cae55c994448-kube-api-access-pnk5x\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.583530 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.583489 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-tls\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.585955 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.585931 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5db6382c-38a2-4cfe-a827-cae55c994448-node-exporter-tls\") pod \"node-exporter-2sx7h\" (UID: \"5db6382c-38a2-4cfe-a827-cae55c994448\") " pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.741830 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.741787 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2sx7h" Jul 13 18:59:46.756809 ip-10-0-133-70 kubenswrapper[2580]: W0713 18:59:46.756772 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db6382c_38a2_4cfe_a827_cae55c994448.slice/crio-3fee7b80a0a00ac17427d4def55b708766528e626a18ccd7acbe118d0cbee441 WatchSource:0}: Error finding container 3fee7b80a0a00ac17427d4def55b708766528e626a18ccd7acbe118d0cbee441: Status 404 returned error can't find the container with id 3fee7b80a0a00ac17427d4def55b708766528e626a18ccd7acbe118d0cbee441 Jul 13 18:59:46.985630 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:46.985553 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 18:59:47.076874 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:47.076843 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sx7h" event={"ID":"5db6382c-38a2-4cfe-a827-cae55c994448","Type":"ContainerStarted","Data":"3fee7b80a0a00ac17427d4def55b708766528e626a18ccd7acbe118d0cbee441"} Jul 13 18:59:48.085309 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:48.085270 2580 generic.go:356] "Generic (PLEG): container finished" podID="5db6382c-38a2-4cfe-a827-cae55c994448" containerID="995c78d2bcac55abcda2cf023ef0a9200ff80d599eb614a35422d7aa7e14ae29" exitCode=0 Jul 13 18:59:48.085715 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:48.085347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sx7h" event={"ID":"5db6382c-38a2-4cfe-a827-cae55c994448","Type":"ContainerDied","Data":"995c78d2bcac55abcda2cf023ef0a9200ff80d599eb614a35422d7aa7e14ae29"} Jul 13 18:59:49.092301 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:49.092266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sx7h" event={"ID":"5db6382c-38a2-4cfe-a827-cae55c994448","Type":"ContainerStarted","Data":"b80d8c7836cb7b3da55a30c3b402d09515a2afc04b52ee4c833086a40b6609f4"} Jul 13 18:59:49.092799 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:49.092331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2sx7h" event={"ID":"5db6382c-38a2-4cfe-a827-cae55c994448","Type":"ContainerStarted","Data":"57ee97767ba9a20805c70f29f6f556aedc07b933e1be1adfa1b8dcb05c5e98fc"} Jul 13 18:59:49.116820 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:49.116761 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2sx7h" podStartSLOduration=3.079079454 podStartE2EDuration="4.116745004s" totalImagesPullingTime="972.470258ms" totalInitContainerRuntime="65.195292ms" isStatefulPod=true podCreationTimestamp="2026-07-13 18:59:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 18:59:49.116109778 +0000 UTC m=+93.144668115" watchObservedRunningTime="2026-07-13 18:59:49.116745004 +0000 UTC m=+93.145303340" Jul 13 18:59:52.112574 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:52.112535 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fb8548b8f-9swm6"] Jul 13 18:59:54.955144 ip-10-0-133-70 kubenswrapper[2580]: I0713 18:59:54.955109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vk9kv" Jul 13 19:00:05.613822 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.613783 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f4fd9f9d-hb44b"] Jul 13 19:00:05.637310 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.637278 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f4fd9f9d-hb44b"] Jul 13 19:00:05.637458 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.637416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.641690 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.641646 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 19:00:05.641937 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.641909 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 19:00:05.642619 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.642604 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 19:00:05.642974 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.642949 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 19:00:05.642974 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.642969 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 19:00:05.643122 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.643000 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 19:00:05.643256 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.643229 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 19:00:05.643567 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.643549 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jrnfq\"" Jul 13 19:00:05.647637 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.647616 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 19:00:05.742308 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-console-config\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.742308 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742312 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-oauth-serving-cert\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.742507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-service-ca\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.742507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-oauth-config\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.742507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742475 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-serving-cert\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.742507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-trusted-ca-bundle\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.742630 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.742518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktqks\" (UniqueName: \"kubernetes.io/projected/b2cc3e3c-0147-427a-8703-96255720e01c-kube-api-access-ktqks\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.842954 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.842904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-console-config\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.842954 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.842959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-oauth-serving-cert\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-service-ca\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-oauth-config\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-serving-cert\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-trusted-ca-bundle\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktqks\" (UniqueName: \"kubernetes.io/projected/b2cc3e3c-0147-427a-8703-96255720e01c-kube-api-access-ktqks\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843861 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843837 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-oauth-serving-cert\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.843861 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843844 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-console-config\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.844027 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.843848 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-service-ca\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.844404 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.844382 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-trusted-ca-bundle\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.845545 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.845526 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-serving-cert\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.845693 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.845647 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-oauth-config\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.859538 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.859504 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktqks\" (UniqueName: \"kubernetes.io/projected/b2cc3e3c-0147-427a-8703-96255720e01c-kube-api-access-ktqks\") pod \"console-56f4fd9f9d-hb44b\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:05.947729 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:05.947595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:06.095051 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:06.095025 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f4fd9f9d-hb44b"] Jul 13 19:00:06.097690 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:00:06.097647 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2cc3e3c_0147_427a_8703_96255720e01c.slice/crio-ad9915e27304127e666fe2884d7ec71694559e71787013e4ce66ddaba8596f57 WatchSource:0}: Error finding container ad9915e27304127e666fe2884d7ec71694559e71787013e4ce66ddaba8596f57: Status 404 returned error can't find the container with id ad9915e27304127e666fe2884d7ec71694559e71787013e4ce66ddaba8596f57 Jul 13 19:00:06.161033 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:06.160993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f4fd9f9d-hb44b" event={"ID":"b2cc3e3c-0147-427a-8703-96255720e01c","Type":"ContainerStarted","Data":"ad9915e27304127e666fe2884d7ec71694559e71787013e4ce66ddaba8596f57"} Jul 13 19:00:10.179782 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:10.179735 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f4fd9f9d-hb44b" event={"ID":"b2cc3e3c-0147-427a-8703-96255720e01c","Type":"ContainerStarted","Data":"9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023"} Jul 13 19:00:10.200277 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:10.200219 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f4fd9f9d-hb44b" podStartSLOduration=2.182850786 podStartE2EDuration="5.200202562s" totalImagesPullingTime="3.017351776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:00:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:00:10.19980762 +0000 UTC m=+114.228365959" watchObservedRunningTime="2026-07-13 19:00:10.200202562 +0000 UTC m=+114.228760898" Jul 13 19:00:15.948363 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:15.948307 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:15.949121 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:15.948408 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:15.953344 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:15.953319 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:16.212107 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:16.212006 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:00:17.133066 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.133023 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" podUID="73b27864-68b8-4b8f-84b2-47dc93109879" containerName="registry" containerID="cri-o://1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b" gracePeriod=30 Jul 13 19:00:17.213774 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.213657 2580 generic.go:356] "Generic (PLEG): container finished" podID="67985222-8a17-45f7-b42c-fa011cdd7fd1" containerID="46b93906891a72b7e541745b81d95e3e0157d70b21fafaebe886ae8e741e3770" exitCode=0 Jul 13 19:00:17.213774 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.213730 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" event={"ID":"67985222-8a17-45f7-b42c-fa011cdd7fd1","Type":"ContainerDied","Data":"46b93906891a72b7e541745b81d95e3e0157d70b21fafaebe886ae8e741e3770"} Jul 13 19:00:17.214148 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.214133 2580 scope.go:117] "RemoveContainer" containerID="46b93906891a72b7e541745b81d95e3e0157d70b21fafaebe886ae8e741e3770" Jul 13 19:00:17.385684 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.385638 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 19:00:17.456087 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456017 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-installation-pull-secrets\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.456292 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456120 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrcck\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-kube-api-access-mrcck\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.456292 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456207 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-trusted-ca\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.456292 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456266 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-image-registry-private-configuration\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.456457 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456292 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-registry-certificates\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.456603 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456574 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-bound-sa-token\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.456784 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456649 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.457153 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.456981 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73b27864-68b8-4b8f-84b2-47dc93109879-ca-trust-extracted\") pod \"73b27864-68b8-4b8f-84b2-47dc93109879\" (UID: \"73b27864-68b8-4b8f-84b2-47dc93109879\") " Jul 13 19:00:17.457153 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.457076 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:17.457153 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.457107 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:00:17.457489 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.457350 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-registry-certificates\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.457489 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.457368 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/73b27864-68b8-4b8f-84b2-47dc93109879-trusted-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.459350 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.459276 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:17.459350 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.459332 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-kube-api-access-mrcck" (OuterVolumeSpecName: "kube-api-access-mrcck") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "kube-api-access-mrcck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:00:17.459655 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.459627 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:00:17.459751 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.459691 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:00:17.459798 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.459782 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:00:17.466377 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.466313 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73b27864-68b8-4b8f-84b2-47dc93109879-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "73b27864-68b8-4b8f-84b2-47dc93109879" (UID: "73b27864-68b8-4b8f-84b2-47dc93109879"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:00:17.558206 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.558159 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-image-registry-private-configuration\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.558206 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.558204 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-bound-sa-token\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.558417 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.558221 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-registry-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.558417 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.558233 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/73b27864-68b8-4b8f-84b2-47dc93109879-ca-trust-extracted\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.558417 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.558247 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/73b27864-68b8-4b8f-84b2-47dc93109879-installation-pull-secrets\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:17.558417 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:17.558261 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrcck\" (UniqueName: \"kubernetes.io/projected/73b27864-68b8-4b8f-84b2-47dc93109879-kube-api-access-mrcck\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:00:18.220160 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.220066 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-sw65h" event={"ID":"67985222-8a17-45f7-b42c-fa011cdd7fd1","Type":"ContainerStarted","Data":"637b1ee2b03dae9a9036800b5a9aef555d2ed8a5db13afd800dc35f2d7bad501"} Jul 13 19:00:18.221380 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.221352 2580 generic.go:356] "Generic (PLEG): container finished" podID="73b27864-68b8-4b8f-84b2-47dc93109879" containerID="1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b" exitCode=0 Jul 13 19:00:18.221517 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.221419 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" Jul 13 19:00:18.221517 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.221431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" event={"ID":"73b27864-68b8-4b8f-84b2-47dc93109879","Type":"ContainerDied","Data":"1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b"} Jul 13 19:00:18.221517 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.221473 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb8548b8f-9swm6" event={"ID":"73b27864-68b8-4b8f-84b2-47dc93109879","Type":"ContainerDied","Data":"3ef8f8822d85f7ceda590b89ef4851a139f1131c632284201aae571e5fbc2006"} Jul 13 19:00:18.221517 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.221501 2580 scope.go:117] "RemoveContainer" containerID="1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b" Jul 13 19:00:18.223077 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.223048 2580 generic.go:356] "Generic (PLEG): container finished" podID="fc33eccb-8c33-46d3-8c49-87ce85e050af" containerID="c959f971538fc80c3cbb41ebca11d623ef5200f4f31024f2abb36c203c300f03" exitCode=0 Jul 13 19:00:18.223168 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.223077 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" event={"ID":"fc33eccb-8c33-46d3-8c49-87ce85e050af","Type":"ContainerDied","Data":"c959f971538fc80c3cbb41ebca11d623ef5200f4f31024f2abb36c203c300f03"} Jul 13 19:00:18.223378 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.223365 2580 scope.go:117] "RemoveContainer" containerID="c959f971538fc80c3cbb41ebca11d623ef5200f4f31024f2abb36c203c300f03" Jul 13 19:00:18.233106 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.233087 2580 scope.go:117] "RemoveContainer" containerID="1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b" Jul 13 19:00:18.233459 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:00:18.233440 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b\": container with ID starting with 1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b not found: ID does not exist" containerID="1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b" Jul 13 19:00:18.233505 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.233469 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b"} err="failed to get container status \"1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b\": rpc error: code = NotFound desc = could not find container \"1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b\": container with ID starting with 1c544755866292fd1b6bd8f08aab250b2a4e3a97d86154ec76011e9655bddd4b not found: ID does not exist" Jul 13 19:00:18.250575 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.250541 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6fb8548b8f-9swm6"] Jul 13 19:00:18.255561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.255529 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6fb8548b8f-9swm6"] Jul 13 19:00:18.545946 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:18.545869 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73b27864-68b8-4b8f-84b2-47dc93109879" path="/var/lib/kubelet/pods/73b27864-68b8-4b8f-84b2-47dc93109879/volumes" Jul 13 19:00:19.229947 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:19.229912 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-44q2g" event={"ID":"fc33eccb-8c33-46d3-8c49-87ce85e050af","Type":"ContainerStarted","Data":"b07fb94027b6cf1a84abaaccb1a9042e3e4f8cc9394119686489e66088cfbd66"} Jul 13 19:00:22.242568 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:22.242481 2580 generic.go:356] "Generic (PLEG): container finished" podID="eaea0df7-d85f-48b1-b100-dacf528c63b2" containerID="9c0cded27ad0ab3b715a270fc9a375b32e1fd1ddc82e5c535dbd0ac26c35c327" exitCode=0 Jul 13 19:00:22.242568 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:22.242528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" event={"ID":"eaea0df7-d85f-48b1-b100-dacf528c63b2","Type":"ContainerDied","Data":"9c0cded27ad0ab3b715a270fc9a375b32e1fd1ddc82e5c535dbd0ac26c35c327"} Jul 13 19:00:22.243012 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:22.242903 2580 scope.go:117] "RemoveContainer" containerID="9c0cded27ad0ab3b715a270fc9a375b32e1fd1ddc82e5c535dbd0ac26c35c327" Jul 13 19:00:23.248360 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:00:23.248318 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-ngr59" event={"ID":"eaea0df7-d85f-48b1-b100-dacf528c63b2","Type":"ContainerStarted","Data":"1df49872fc2ad424d6abc7eeb96ee8994e6fe5af10d9f0740b1fda237af9d39c"} Jul 13 19:01:35.452512 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.452367 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68cdd6b489-f5fdr"] Jul 13 19:01:35.453136 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.452843 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73b27864-68b8-4b8f-84b2-47dc93109879" containerName="registry" Jul 13 19:01:35.453136 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.452864 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="73b27864-68b8-4b8f-84b2-47dc93109879" containerName="registry" Jul 13 19:01:35.453136 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.452985 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="73b27864-68b8-4b8f-84b2-47dc93109879" containerName="registry" Jul 13 19:01:35.457313 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.457289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.468217 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.468188 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68cdd6b489-f5fdr"] Jul 13 19:01:35.518284 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518239 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7v4n\" (UniqueName: \"kubernetes.io/projected/bfadfe9d-ace3-40fe-a138-818ae8e638f3-kube-api-access-z7v4n\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.518284 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-oauth-config\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.518492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-service-ca\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.518492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-trusted-ca-bundle\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.518492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-config\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.518492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-oauth-serving-cert\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.518492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.518428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-serving-cert\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619044 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-oauth-config\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619044 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-service-ca\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619264 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619073 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-trusted-ca-bundle\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619264 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-config\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619264 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619134 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-oauth-serving-cert\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619264 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-serving-cert\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619264 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7v4n\" (UniqueName: \"kubernetes.io/projected/bfadfe9d-ace3-40fe-a138-818ae8e638f3-kube-api-access-z7v4n\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.619909 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619883 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-service-ca\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.620026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.619996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-config\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.620071 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.620051 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-oauth-serving-cert\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.620403 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.620384 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-trusted-ca-bundle\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.621631 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.621610 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-oauth-config\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.621807 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.621782 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-serving-cert\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.627679 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.627640 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7v4n\" (UniqueName: \"kubernetes.io/projected/bfadfe9d-ace3-40fe-a138-818ae8e638f3-kube-api-access-z7v4n\") pod \"console-68cdd6b489-f5fdr\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.767507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.767416 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:35.912632 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:35.909395 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68cdd6b489-f5fdr"] Jul 13 19:01:36.528532 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:36.528492 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68cdd6b489-f5fdr" event={"ID":"bfadfe9d-ace3-40fe-a138-818ae8e638f3","Type":"ContainerStarted","Data":"9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec"} Jul 13 19:01:36.529006 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:36.528551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68cdd6b489-f5fdr" event={"ID":"bfadfe9d-ace3-40fe-a138-818ae8e638f3","Type":"ContainerStarted","Data":"e35f46e479c38227de296380809caace50754f884962712a28910a4b23a1403e"} Jul 13 19:01:36.547091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:36.547041 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68cdd6b489-f5fdr" podStartSLOduration=1.547023962 podStartE2EDuration="1.547023962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:01:36.546377898 +0000 UTC m=+200.574936236" watchObservedRunningTime="2026-07-13 19:01:36.547023962 +0000 UTC m=+200.575582299" Jul 13 19:01:45.767732 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:45.767689 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:45.768217 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:45.767768 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:45.772851 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:45.772825 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:46.575640 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:46.575606 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:01:46.625564 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:01:46.625516 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f4fd9f9d-hb44b"] Jul 13 19:02:11.653865 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.653804 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56f4fd9f9d-hb44b" podUID="b2cc3e3c-0147-427a-8703-96255720e01c" containerName="console" containerID="cri-o://9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023" gracePeriod=15 Jul 13 19:02:11.735901 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.735858 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm"] Jul 13 19:02:11.739195 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.739171 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.741975 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.741943 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h9f6k\"" Jul 13 19:02:11.741975 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.741973 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 19:02:11.742854 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.742831 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 19:02:11.747727 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.747704 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm"] Jul 13 19:02:11.798504 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.798451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tx4kc\" (UniqueName: \"kubernetes.io/projected/6f26afb9-bc89-440d-aed5-d472ab39b02a-kube-api-access-tx4kc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.798717 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.798539 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.798717 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.798599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.899214 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.899182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.899358 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.899257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tx4kc\" (UniqueName: \"kubernetes.io/projected/6f26afb9-bc89-440d-aed5-d472ab39b02a-kube-api-access-tx4kc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.899358 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.899285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.899581 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.899566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.899617 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.899602 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:11.904418 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.904360 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:02:11.908071 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:11.908045 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tx4kc\" (UniqueName: \"kubernetes.io/projected/6f26afb9-bc89-440d-aed5-d472ab39b02a-kube-api-access-tx4kc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:12.000317 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000284 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-service-ca\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000317 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000323 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-oauth-serving-cert\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000353 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-serving-cert\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000380 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-trusted-ca-bundle\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000433 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-oauth-config\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000469 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktqks\" (UniqueName: \"kubernetes.io/projected/b2cc3e3c-0147-427a-8703-96255720e01c-kube-api-access-ktqks\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000519 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-console-config\") pod \"b2cc3e3c-0147-427a-8703-96255720e01c\" (UID: \"b2cc3e3c-0147-427a-8703-96255720e01c\") " Jul 13 19:02:12.000885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000855 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-service-ca" (OuterVolumeSpecName: "service-ca") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:02:12.000949 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.000876 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:02:12.001045 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.001019 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:02:12.001116 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.001087 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-console-config" (OuterVolumeSpecName: "console-config") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:02:12.002885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.002864 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:02:12.002960 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.002908 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2cc3e3c-0147-427a-8703-96255720e01c-kube-api-access-ktqks" (OuterVolumeSpecName: "kube-api-access-ktqks") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "kube-api-access-ktqks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:02:12.003019 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.003003 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b2cc3e3c-0147-427a-8703-96255720e01c" (UID: "b2cc3e3c-0147-427a-8703-96255720e01c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:02:12.049963 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.049927 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:12.101583 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101549 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.101583 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101584 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-oauth-serving-cert\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.101818 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101599 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-serving-cert\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.101818 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101613 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-trusted-ca-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.101818 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101626 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b2cc3e3c-0147-427a-8703-96255720e01c-console-oauth-config\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.101818 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101638 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktqks\" (UniqueName: \"kubernetes.io/projected/b2cc3e3c-0147-427a-8703-96255720e01c-kube-api-access-ktqks\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.101818 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.101653 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b2cc3e3c-0147-427a-8703-96255720e01c-console-config\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:12.181299 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.181275 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm"] Jul 13 19:02:12.183330 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:02:12.183306 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f26afb9_bc89_440d_aed5_d472ab39b02a.slice/crio-e66b50ddc2661094fbe45e14a6ff96c61e949dd125fd9fc08cb165b79f47788a WatchSource:0}: Error finding container e66b50ddc2661094fbe45e14a6ff96c61e949dd125fd9fc08cb165b79f47788a: Status 404 returned error can't find the container with id e66b50ddc2661094fbe45e14a6ff96c61e949dd125fd9fc08cb165b79f47788a Jul 13 19:02:12.668899 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.668861 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" event={"ID":"6f26afb9-bc89-440d-aed5-d472ab39b02a","Type":"ContainerStarted","Data":"e66b50ddc2661094fbe45e14a6ff96c61e949dd125fd9fc08cb165b79f47788a"} Jul 13 19:02:12.670638 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.670603 2580 generic.go:356] "Generic (PLEG): container finished" podID="b2cc3e3c-0147-427a-8703-96255720e01c" containerID="9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023" exitCode=0 Jul 13 19:02:12.670807 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.670648 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f4fd9f9d-hb44b" event={"ID":"b2cc3e3c-0147-427a-8703-96255720e01c","Type":"ContainerDied","Data":"9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023"} Jul 13 19:02:12.670807 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.670710 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f4fd9f9d-hb44b" Jul 13 19:02:12.670807 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.670727 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f4fd9f9d-hb44b" event={"ID":"b2cc3e3c-0147-427a-8703-96255720e01c","Type":"ContainerDied","Data":"ad9915e27304127e666fe2884d7ec71694559e71787013e4ce66ddaba8596f57"} Jul 13 19:02:12.670807 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.670765 2580 scope.go:117] "RemoveContainer" containerID="9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023" Jul 13 19:02:12.679523 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.679504 2580 scope.go:117] "RemoveContainer" containerID="9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023" Jul 13 19:02:12.679887 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:02:12.679868 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023\": container with ID starting with 9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023 not found: ID does not exist" containerID="9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023" Jul 13 19:02:12.679952 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.679898 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023"} err="failed to get container status \"9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023\": rpc error: code = NotFound desc = could not find container \"9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023\": container with ID starting with 9e724678cff93bc50b665b3b0f7877fe9ce4d2780a8f1441f8e49a252d43d023 not found: ID does not exist" Jul 13 19:02:12.686351 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.686321 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f4fd9f9d-hb44b"] Jul 13 19:02:12.690243 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:12.690213 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f4fd9f9d-hb44b"] Jul 13 19:02:14.547151 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:14.547112 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2cc3e3c-0147-427a-8703-96255720e01c" path="/var/lib/kubelet/pods/b2cc3e3c-0147-427a-8703-96255720e01c/volumes" Jul 13 19:02:20.707190 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:20.707150 2580 generic.go:356] "Generic (PLEG): container finished" podID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerID="d448044e53f4a611ac480565e2b88cdbe939823489e858f4b4913bf05aed3918" exitCode=0 Jul 13 19:02:20.707609 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:20.707238 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" event={"ID":"6f26afb9-bc89-440d-aed5-d472ab39b02a","Type":"ContainerDied","Data":"d448044e53f4a611ac480565e2b88cdbe939823489e858f4b4913bf05aed3918"} Jul 13 19:02:23.720941 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:23.720907 2580 generic.go:356] "Generic (PLEG): container finished" podID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerID="a0322698d446644a586fe5f52758b83762989922439190ae0aa9f8bf6bfefe15" exitCode=0 Jul 13 19:02:23.721394 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:23.720997 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" event={"ID":"6f26afb9-bc89-440d-aed5-d472ab39b02a","Type":"ContainerDied","Data":"a0322698d446644a586fe5f52758b83762989922439190ae0aa9f8bf6bfefe15"} Jul 13 19:02:32.759098 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:32.759056 2580 generic.go:356] "Generic (PLEG): container finished" podID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerID="602baf50ff9593f7713e38da55a2666c721c5c75574bddfb5b97aa7e1edcd9f3" exitCode=0 Jul 13 19:02:32.759556 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:32.759167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" event={"ID":"6f26afb9-bc89-440d-aed5-d472ab39b02a","Type":"ContainerDied","Data":"602baf50ff9593f7713e38da55a2666c721c5c75574bddfb5b97aa7e1edcd9f3"} Jul 13 19:02:33.920556 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:33.920518 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:34.096407 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.096371 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-bundle\") pod \"6f26afb9-bc89-440d-aed5-d472ab39b02a\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " Jul 13 19:02:34.096577 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.096454 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-util\") pod \"6f26afb9-bc89-440d-aed5-d472ab39b02a\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " Jul 13 19:02:34.096577 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.096484 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tx4kc\" (UniqueName: \"kubernetes.io/projected/6f26afb9-bc89-440d-aed5-d472ab39b02a-kube-api-access-tx4kc\") pod \"6f26afb9-bc89-440d-aed5-d472ab39b02a\" (UID: \"6f26afb9-bc89-440d-aed5-d472ab39b02a\") " Jul 13 19:02:34.097168 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.097136 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-bundle" (OuterVolumeSpecName: "bundle") pod "6f26afb9-bc89-440d-aed5-d472ab39b02a" (UID: "6f26afb9-bc89-440d-aed5-d472ab39b02a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:02:34.098892 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.098869 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f26afb9-bc89-440d-aed5-d472ab39b02a-kube-api-access-tx4kc" (OuterVolumeSpecName: "kube-api-access-tx4kc") pod "6f26afb9-bc89-440d-aed5-d472ab39b02a" (UID: "6f26afb9-bc89-440d-aed5-d472ab39b02a"). InnerVolumeSpecName "kube-api-access-tx4kc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:02:34.101131 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.101097 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-util" (OuterVolumeSpecName: "util") pod "6f26afb9-bc89-440d-aed5-d472ab39b02a" (UID: "6f26afb9-bc89-440d-aed5-d472ab39b02a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:02:34.197482 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.197442 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:34.197482 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.197476 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tx4kc\" (UniqueName: \"kubernetes.io/projected/6f26afb9-bc89-440d-aed5-d472ab39b02a-kube-api-access-tx4kc\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:34.197482 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.197485 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6f26afb9-bc89-440d-aed5-d472ab39b02a-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:34.769129 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.769096 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" event={"ID":"6f26afb9-bc89-440d-aed5-d472ab39b02a","Type":"ContainerDied","Data":"e66b50ddc2661094fbe45e14a6ff96c61e949dd125fd9fc08cb165b79f47788a"} Jul 13 19:02:34.769129 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.769115 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540sftjm" Jul 13 19:02:34.769340 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:34.769144 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e66b50ddc2661094fbe45e14a6ff96c61e949dd125fd9fc08cb165b79f47788a" Jul 13 19:02:39.720468 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720381 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h"] Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720717 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="util" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720730 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="util" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720740 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2cc3e3c-0147-427a-8703-96255720e01c" containerName="console" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720745 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2cc3e3c-0147-427a-8703-96255720e01c" containerName="console" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720754 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="extract" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720760 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="extract" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720775 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="pull" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720780 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="pull" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720841 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2cc3e3c-0147-427a-8703-96255720e01c" containerName="console" Jul 13 19:02:39.720964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.720850 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f26afb9-bc89-440d-aed5-d472ab39b02a" containerName="extract" Jul 13 19:02:39.723090 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.723074 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:39.725548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.725521 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qhrgh\"" Jul 13 19:02:39.725694 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.725567 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 13 19:02:39.725694 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.725526 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:02:39.734820 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.734794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h"] Jul 13 19:02:39.737522 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.737493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbbdc5d8-c03b-459b-a702-ba461ddf881f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dmf5h\" (UID: \"fbbdc5d8-c03b-459b-a702-ba461ddf881f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:39.737630 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.737545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqj8h\" (UniqueName: \"kubernetes.io/projected/fbbdc5d8-c03b-459b-a702-ba461ddf881f-kube-api-access-dqj8h\") pod \"cert-manager-operator-controller-manager-55c444bf86-dmf5h\" (UID: \"fbbdc5d8-c03b-459b-a702-ba461ddf881f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:39.838101 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.838058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbbdc5d8-c03b-459b-a702-ba461ddf881f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dmf5h\" (UID: \"fbbdc5d8-c03b-459b-a702-ba461ddf881f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:39.838284 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.838147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqj8h\" (UniqueName: \"kubernetes.io/projected/fbbdc5d8-c03b-459b-a702-ba461ddf881f-kube-api-access-dqj8h\") pod \"cert-manager-operator-controller-manager-55c444bf86-dmf5h\" (UID: \"fbbdc5d8-c03b-459b-a702-ba461ddf881f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:39.838540 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.838521 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fbbdc5d8-c03b-459b-a702-ba461ddf881f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-dmf5h\" (UID: \"fbbdc5d8-c03b-459b-a702-ba461ddf881f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:39.848440 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:39.848404 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqj8h\" (UniqueName: \"kubernetes.io/projected/fbbdc5d8-c03b-459b-a702-ba461ddf881f-kube-api-access-dqj8h\") pod \"cert-manager-operator-controller-manager-55c444bf86-dmf5h\" (UID: \"fbbdc5d8-c03b-459b-a702-ba461ddf881f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:40.031649 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:40.031554 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" Jul 13 19:02:40.174307 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:40.174167 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h"] Jul 13 19:02:40.177568 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:02:40.177534 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbdc5d8_c03b_459b_a702_ba461ddf881f.slice/crio-fdf436e41b91151f7da547360bdd910d34459965d74dfc5b0886e48677b71035 WatchSource:0}: Error finding container fdf436e41b91151f7da547360bdd910d34459965d74dfc5b0886e48677b71035: Status 404 returned error can't find the container with id fdf436e41b91151f7da547360bdd910d34459965d74dfc5b0886e48677b71035 Jul 13 19:02:40.794185 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:40.794147 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" event={"ID":"fbbdc5d8-c03b-459b-a702-ba461ddf881f","Type":"ContainerStarted","Data":"fdf436e41b91151f7da547360bdd910d34459965d74dfc5b0886e48677b71035"} Jul 13 19:02:42.805602 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:42.805559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" event={"ID":"fbbdc5d8-c03b-459b-a702-ba461ddf881f","Type":"ContainerStarted","Data":"2410a4780ca264744bf008762bb87589f570eaf8443752068b23a5332feaf781"} Jul 13 19:02:42.828557 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:42.828499 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-dmf5h" podStartSLOduration=2.185985688 podStartE2EDuration="3.828478834s" totalImagesPullingTime="1.642493146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:02:42.826876339 +0000 UTC m=+266.855434694" watchObservedRunningTime="2026-07-13 19:02:42.828478834 +0000 UTC m=+266.857037171" Jul 13 19:02:43.787492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.787455 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms"] Jul 13 19:02:43.789904 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.789879 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:43.792701 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.792662 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 19:02:43.792833 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.792685 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 19:02:43.793508 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.793491 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h9f6k\"" Jul 13 19:02:43.802556 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.802526 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms"] Jul 13 19:02:43.966118 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.966075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:43.966118 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.966119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxx88\" (UniqueName: \"kubernetes.io/projected/4c4daebe-5b83-4518-a615-6c559410d1ea-kube-api-access-cxx88\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:43.966538 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:43.966225 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.067162 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.067052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.067162 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.067115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.067162 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.067137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxx88\" (UniqueName: \"kubernetes.io/projected/4c4daebe-5b83-4518-a615-6c559410d1ea-kube-api-access-cxx88\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.067529 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.067504 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.067591 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.067515 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.079152 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.079105 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxx88\" (UniqueName: \"kubernetes.io/projected/4c4daebe-5b83-4518-a615-6c559410d1ea-kube-api-access-cxx88\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.099919 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.099883 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:44.234733 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.234690 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms"] Jul 13 19:02:44.237889 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:02:44.237859 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4daebe_5b83_4518_a615_6c559410d1ea.slice/crio-9e56a4017b1cbf9be43425c958a09fbc2ee8cb0aaa0598ca3ddddae9c6bd5cd9 WatchSource:0}: Error finding container 9e56a4017b1cbf9be43425c958a09fbc2ee8cb0aaa0598ca3ddddae9c6bd5cd9: Status 404 returned error can't find the container with id 9e56a4017b1cbf9be43425c958a09fbc2ee8cb0aaa0598ca3ddddae9c6bd5cd9 Jul 13 19:02:44.815405 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.815316 2580 generic.go:356] "Generic (PLEG): container finished" podID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerID="9d669cb3e9c3465d4ff89dadd1386d149de032bbd9a8f8d0d619a2238e6b7109" exitCode=0 Jul 13 19:02:44.815405 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.815371 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" event={"ID":"4c4daebe-5b83-4518-a615-6c559410d1ea","Type":"ContainerDied","Data":"9d669cb3e9c3465d4ff89dadd1386d149de032bbd9a8f8d0d619a2238e6b7109"} Jul 13 19:02:44.815581 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:44.815448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" event={"ID":"4c4daebe-5b83-4518-a615-6c559410d1ea","Type":"ContainerStarted","Data":"9e56a4017b1cbf9be43425c958a09fbc2ee8cb0aaa0598ca3ddddae9c6bd5cd9"} Jul 13 19:02:46.305903 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.305869 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-g2c26"] Jul 13 19:02:46.308457 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.308432 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.310884 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.310845 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 13 19:02:46.311026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.310981 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 13 19:02:46.311871 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.311850 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-8jjr6\"" Jul 13 19:02:46.318116 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.318089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-g2c26"] Jul 13 19:02:46.385020 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.384977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a30e553-0d9c-493b-95cd-57356ecee4ff-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-g2c26\" (UID: \"8a30e553-0d9c-493b-95cd-57356ecee4ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.385020 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.385025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9fsc\" (UniqueName: \"kubernetes.io/projected/8a30e553-0d9c-493b-95cd-57356ecee4ff-kube-api-access-b9fsc\") pod \"cert-manager-cainjector-559cdd8c87-g2c26\" (UID: \"8a30e553-0d9c-493b-95cd-57356ecee4ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.486030 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.485485 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a30e553-0d9c-493b-95cd-57356ecee4ff-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-g2c26\" (UID: \"8a30e553-0d9c-493b-95cd-57356ecee4ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.486030 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.485543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9fsc\" (UniqueName: \"kubernetes.io/projected/8a30e553-0d9c-493b-95cd-57356ecee4ff-kube-api-access-b9fsc\") pod \"cert-manager-cainjector-559cdd8c87-g2c26\" (UID: \"8a30e553-0d9c-493b-95cd-57356ecee4ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.494852 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.494811 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8a30e553-0d9c-493b-95cd-57356ecee4ff-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-g2c26\" (UID: \"8a30e553-0d9c-493b-95cd-57356ecee4ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.495068 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.495045 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9fsc\" (UniqueName: \"kubernetes.io/projected/8a30e553-0d9c-493b-95cd-57356ecee4ff-kube-api-access-b9fsc\") pod \"cert-manager-cainjector-559cdd8c87-g2c26\" (UID: \"8a30e553-0d9c-493b-95cd-57356ecee4ff\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:46.626587 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:46.626546 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" Jul 13 19:02:47.182804 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:47.182778 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-g2c26"] Jul 13 19:02:47.186274 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:02:47.186239 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a30e553_0d9c_493b_95cd_57356ecee4ff.slice/crio-b0ec9b51406b6312a8d22f0e113113d208c06be0fd144a674be7229d34d37396 WatchSource:0}: Error finding container b0ec9b51406b6312a8d22f0e113113d208c06be0fd144a674be7229d34d37396: Status 404 returned error can't find the container with id b0ec9b51406b6312a8d22f0e113113d208c06be0fd144a674be7229d34d37396 Jul 13 19:02:47.830938 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:47.830905 2580 generic.go:356] "Generic (PLEG): container finished" podID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerID="0042f02299a40d092b0a68ca66d1ac9e90bfd8a033acbd6c38bdaf5225bc1ef2" exitCode=0 Jul 13 19:02:47.831424 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:47.830994 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" event={"ID":"4c4daebe-5b83-4518-a615-6c559410d1ea","Type":"ContainerDied","Data":"0042f02299a40d092b0a68ca66d1ac9e90bfd8a033acbd6c38bdaf5225bc1ef2"} Jul 13 19:02:47.832254 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:47.832232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" event={"ID":"8a30e553-0d9c-493b-95cd-57356ecee4ff","Type":"ContainerStarted","Data":"b0ec9b51406b6312a8d22f0e113113d208c06be0fd144a674be7229d34d37396"} Jul 13 19:02:48.838876 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:48.838835 2580 generic.go:356] "Generic (PLEG): container finished" podID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerID="87cf722797b11ed968320e5329f1b7ad3e3e99a53107667c478752a919744584" exitCode=0 Jul 13 19:02:48.839273 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:48.838913 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" event={"ID":"4c4daebe-5b83-4518-a615-6c559410d1ea","Type":"ContainerDied","Data":"87cf722797b11ed968320e5329f1b7ad3e3e99a53107667c478752a919744584"} Jul 13 19:02:50.796555 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.796515 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:50.823918 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.823889 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxx88\" (UniqueName: \"kubernetes.io/projected/4c4daebe-5b83-4518-a615-6c559410d1ea-kube-api-access-cxx88\") pod \"4c4daebe-5b83-4518-a615-6c559410d1ea\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " Jul 13 19:02:50.824063 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.823936 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-bundle\") pod \"4c4daebe-5b83-4518-a615-6c559410d1ea\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " Jul 13 19:02:50.824063 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.823976 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-util\") pod \"4c4daebe-5b83-4518-a615-6c559410d1ea\" (UID: \"4c4daebe-5b83-4518-a615-6c559410d1ea\") " Jul 13 19:02:50.824407 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.824374 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-bundle" (OuterVolumeSpecName: "bundle") pod "4c4daebe-5b83-4518-a615-6c559410d1ea" (UID: "4c4daebe-5b83-4518-a615-6c559410d1ea"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:02:50.826250 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.826221 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4daebe-5b83-4518-a615-6c559410d1ea-kube-api-access-cxx88" (OuterVolumeSpecName: "kube-api-access-cxx88") pod "4c4daebe-5b83-4518-a615-6c559410d1ea" (UID: "4c4daebe-5b83-4518-a615-6c559410d1ea"). InnerVolumeSpecName "kube-api-access-cxx88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:02:50.832001 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.831945 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-util" (OuterVolumeSpecName: "util") pod "4c4daebe-5b83-4518-a615-6c559410d1ea" (UID: "4c4daebe-5b83-4518-a615-6c559410d1ea"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:02:50.854638 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.854601 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" event={"ID":"4c4daebe-5b83-4518-a615-6c559410d1ea","Type":"ContainerDied","Data":"9e56a4017b1cbf9be43425c958a09fbc2ee8cb0aaa0598ca3ddddae9c6bd5cd9"} Jul 13 19:02:50.854638 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.854633 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f8hkms" Jul 13 19:02:50.854843 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.854660 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e56a4017b1cbf9be43425c958a09fbc2ee8cb0aaa0598ca3ddddae9c6bd5cd9" Jul 13 19:02:50.925331 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.925290 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxx88\" (UniqueName: \"kubernetes.io/projected/4c4daebe-5b83-4518-a615-6c559410d1ea-kube-api-access-cxx88\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:50.925331 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.925325 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:50.925331 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:50.925336 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c4daebe-5b83-4518-a615-6c559410d1ea-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:02:51.860878 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:51.860842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" event={"ID":"8a30e553-0d9c-493b-95cd-57356ecee4ff","Type":"ContainerStarted","Data":"01743f06addaf5e564b01f0a24202efe7ecde2598b04019d8bfd05f737fc6813"} Jul 13 19:02:56.330260 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.330206 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-g2c26" podStartSLOduration=6.703722872 podStartE2EDuration="10.330189146s" totalImagesPullingTime="3.626466274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:02:51.875726936 +0000 UTC m=+275.904285289" watchObservedRunningTime="2026-07-13 19:02:56.330189146 +0000 UTC m=+280.358747482" Jul 13 19:02:56.331745 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.331719 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6v98x"] Jul 13 19:02:56.332146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332126 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="extract" Jul 13 19:02:56.332146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332148 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="extract" Jul 13 19:02:56.332294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332170 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="util" Jul 13 19:02:56.332294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332178 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="util" Jul 13 19:02:56.332294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332199 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="pull" Jul 13 19:02:56.332294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332207 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="pull" Jul 13 19:02:56.332294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.332290 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c4daebe-5b83-4518-a615-6c559410d1ea" containerName="extract" Jul 13 19:02:56.336157 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.336132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.338886 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.338859 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hm97v\"" Jul 13 19:02:56.347256 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.347222 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6v98x"] Jul 13 19:02:56.369176 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.369134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qxb6\" (UniqueName: \"kubernetes.io/projected/5d841c74-966d-44e4-a028-0ecdde47f828-kube-api-access-7qxb6\") pod \"cert-manager-55b4d954d5-6v98x\" (UID: \"5d841c74-966d-44e4-a028-0ecdde47f828\") " pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.369348 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.369186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d841c74-966d-44e4-a028-0ecdde47f828-bound-sa-token\") pod \"cert-manager-55b4d954d5-6v98x\" (UID: \"5d841c74-966d-44e4-a028-0ecdde47f828\") " pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.469896 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.469851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qxb6\" (UniqueName: \"kubernetes.io/projected/5d841c74-966d-44e4-a028-0ecdde47f828-kube-api-access-7qxb6\") pod \"cert-manager-55b4d954d5-6v98x\" (UID: \"5d841c74-966d-44e4-a028-0ecdde47f828\") " pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.469896 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.469905 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d841c74-966d-44e4-a028-0ecdde47f828-bound-sa-token\") pod \"cert-manager-55b4d954d5-6v98x\" (UID: \"5d841c74-966d-44e4-a028-0ecdde47f828\") " pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.478744 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.478710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5d841c74-966d-44e4-a028-0ecdde47f828-bound-sa-token\") pod \"cert-manager-55b4d954d5-6v98x\" (UID: \"5d841c74-966d-44e4-a028-0ecdde47f828\") " pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.478879 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.478830 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qxb6\" (UniqueName: \"kubernetes.io/projected/5d841c74-966d-44e4-a028-0ecdde47f828-kube-api-access-7qxb6\") pod \"cert-manager-55b4d954d5-6v98x\" (UID: \"5d841c74-966d-44e4-a028-0ecdde47f828\") " pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.645325 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.645236 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-6v98x" Jul 13 19:02:56.789726 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.789700 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-6v98x"] Jul 13 19:02:56.792018 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:02:56.791990 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5d841c74_966d_44e4_a028_0ecdde47f828.slice/crio-2f668c832095c64fca8aca0e62ae0712cdae3fef8eb35d8da0dbc455a1840d47 WatchSource:0}: Error finding container 2f668c832095c64fca8aca0e62ae0712cdae3fef8eb35d8da0dbc455a1840d47: Status 404 returned error can't find the container with id 2f668c832095c64fca8aca0e62ae0712cdae3fef8eb35d8da0dbc455a1840d47 Jul 13 19:02:56.889337 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.889299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6v98x" event={"ID":"5d841c74-966d-44e4-a028-0ecdde47f828","Type":"ContainerStarted","Data":"9b03dd4e85cedbeb3c14b87074a83f9e08a831ed41234960aa860cdee432771b"} Jul 13 19:02:56.889521 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.889349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-6v98x" event={"ID":"5d841c74-966d-44e4-a028-0ecdde47f828","Type":"ContainerStarted","Data":"2f668c832095c64fca8aca0e62ae0712cdae3fef8eb35d8da0dbc455a1840d47"} Jul 13 19:02:56.907051 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:02:56.906941 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-6v98x" podStartSLOduration=0.906921126 podStartE2EDuration="906.921126ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:02:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:02:56.905617741 +0000 UTC m=+280.934176079" watchObservedRunningTime="2026-07-13 19:02:56.906921126 +0000 UTC m=+280.935479462" Jul 13 19:03:03.684917 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.684871 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5"] Jul 13 19:03:03.732418 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.732382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5"] Jul 13 19:03:03.732604 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.732501 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.735126 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.735102 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h9f6k\"" Jul 13 19:03:03.735126 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.735120 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 19:03:03.736072 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.736050 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 19:03:03.834126 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.834092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.834344 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.834162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sr2rr\" (UniqueName: \"kubernetes.io/projected/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-kube-api-access-sr2rr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.834344 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.834246 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.935217 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.935114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sr2rr\" (UniqueName: \"kubernetes.io/projected/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-kube-api-access-sr2rr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.935217 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.935162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.935423 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.935287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.935525 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.935509 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.935634 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.935617 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:03.944520 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:03.944484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sr2rr\" (UniqueName: \"kubernetes.io/projected/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-kube-api-access-sr2rr\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:04.042437 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:04.042387 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:04.185037 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:04.184997 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5"] Jul 13 19:03:04.188206 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:03:04.188168 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aaa59ac_f633_4b9a_8e1b_691f891dc0a2.slice/crio-56cc02fa8ac70c98dd692c2791762a798040a721537349d1c6fed7566ba49176 WatchSource:0}: Error finding container 56cc02fa8ac70c98dd692c2791762a798040a721537349d1c6fed7566ba49176: Status 404 returned error can't find the container with id 56cc02fa8ac70c98dd692c2791762a798040a721537349d1c6fed7566ba49176 Jul 13 19:03:04.927576 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:04.927536 2580 generic.go:356] "Generic (PLEG): container finished" podID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerID="3badf712401b53005a3c8c7e91369d22ff37a1a9f82f13b14d5bd835c0a0d6af" exitCode=0 Jul 13 19:03:04.928064 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:04.927622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" event={"ID":"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2","Type":"ContainerDied","Data":"3badf712401b53005a3c8c7e91369d22ff37a1a9f82f13b14d5bd835c0a0d6af"} Jul 13 19:03:04.928064 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:04.927696 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" event={"ID":"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2","Type":"ContainerStarted","Data":"56cc02fa8ac70c98dd692c2791762a798040a721537349d1c6fed7566ba49176"} Jul 13 19:03:05.935920 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:05.934749 2580 generic.go:356] "Generic (PLEG): container finished" podID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerID="0fc77de61bda5f9cfbe33002d5b9d23cfd13d246e172e434d8bda7400eb2c703" exitCode=0 Jul 13 19:03:05.935920 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:05.934941 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" event={"ID":"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2","Type":"ContainerDied","Data":"0fc77de61bda5f9cfbe33002d5b9d23cfd13d246e172e434d8bda7400eb2c703"} Jul 13 19:03:06.941433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:06.941389 2580 generic.go:356] "Generic (PLEG): container finished" podID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerID="ae2892d647bdb686805daa513dc2aba31035f2e859f0df9003ce43e8ac8ed7f5" exitCode=0 Jul 13 19:03:06.941867 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:06.941475 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" event={"ID":"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2","Type":"ContainerDied","Data":"ae2892d647bdb686805daa513dc2aba31035f2e859f0df9003ce43e8ac8ed7f5"} Jul 13 19:03:08.083224 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.083186 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:08.170887 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.170851 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-bundle\") pod \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " Jul 13 19:03:08.171092 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.170904 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-util\") pod \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " Jul 13 19:03:08.171092 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.171006 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sr2rr\" (UniqueName: \"kubernetes.io/projected/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-kube-api-access-sr2rr\") pod \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\" (UID: \"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2\") " Jul 13 19:03:08.171650 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.171617 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-bundle" (OuterVolumeSpecName: "bundle") pod "3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" (UID: "3aaa59ac-f633-4b9a-8e1b-691f891dc0a2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:08.173149 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.173121 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-kube-api-access-sr2rr" (OuterVolumeSpecName: "kube-api-access-sr2rr") pod "3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" (UID: "3aaa59ac-f633-4b9a-8e1b-691f891dc0a2"). InnerVolumeSpecName "kube-api-access-sr2rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:03:08.176532 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.176490 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-util" (OuterVolumeSpecName: "util") pod "3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" (UID: "3aaa59ac-f633-4b9a-8e1b-691f891dc0a2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:03:08.271826 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.271714 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:03:08.271826 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.271761 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:03:08.271826 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.271772 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sr2rr\" (UniqueName: \"kubernetes.io/projected/3aaa59ac-f633-4b9a-8e1b-691f891dc0a2-kube-api-access-sr2rr\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:03:08.952665 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.952635 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" Jul 13 19:03:08.952665 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.952634 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af95mb5" event={"ID":"3aaa59ac-f633-4b9a-8e1b-691f891dc0a2","Type":"ContainerDied","Data":"56cc02fa8ac70c98dd692c2791762a798040a721537349d1c6fed7566ba49176"} Jul 13 19:03:08.952886 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:08.952724 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="56cc02fa8ac70c98dd692c2791762a798040a721537349d1c6fed7566ba49176" Jul 13 19:03:16.406638 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:16.406607 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:03:16.410231 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:16.410197 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:03:16.412354 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:16.412318 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:03:16.416295 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:16.416270 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:03:16.417622 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:16.417599 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 19:03:18.770834 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.770792 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8"] Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771147 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="extract" Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771160 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="extract" Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771170 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="util" Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771176 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="util" Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771189 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="pull" Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771195 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="pull" Jul 13 19:03:18.772253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.771248 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3aaa59ac-f633-4b9a-8e1b-691f891dc0a2" containerName="extract" Jul 13 19:03:18.773219 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.773201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.780905 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.780868 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 13 19:03:18.781086 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.781041 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-c4h5v\"" Jul 13 19:03:18.781508 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.781082 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 13 19:03:18.781931 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.781905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 13 19:03:18.782033 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.781940 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 13 19:03:18.787000 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.786981 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 13 19:03:18.806036 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.806003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8"] Jul 13 19:03:18.860613 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.860574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf82596-42ad-45e4-b53d-352af8564b3b-cert\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.860613 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.860620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3bf82596-42ad-45e4-b53d-352af8564b3b-manager-config\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.860890 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.860759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3bf82596-42ad-45e4-b53d-352af8564b3b-metrics-cert\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.860890 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.860795 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-774tv\" (UniqueName: \"kubernetes.io/projected/3bf82596-42ad-45e4-b53d-352af8564b3b-kube-api-access-774tv\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.961897 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.961853 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf82596-42ad-45e4-b53d-352af8564b3b-cert\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.961897 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.961901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3bf82596-42ad-45e4-b53d-352af8564b3b-manager-config\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.962147 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.961950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3bf82596-42ad-45e4-b53d-352af8564b3b-metrics-cert\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.962147 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.961968 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-774tv\" (UniqueName: \"kubernetes.io/projected/3bf82596-42ad-45e4-b53d-352af8564b3b-kube-api-access-774tv\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.962635 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.962606 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3bf82596-42ad-45e4-b53d-352af8564b3b-manager-config\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.964423 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.964402 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3bf82596-42ad-45e4-b53d-352af8564b3b-metrics-cert\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.964522 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.964433 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bf82596-42ad-45e4-b53d-352af8564b3b-cert\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:18.974432 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:18.974402 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-774tv\" (UniqueName: \"kubernetes.io/projected/3bf82596-42ad-45e4-b53d-352af8564b3b-kube-api-access-774tv\") pod \"lws-controller-manager-64bdc88f66-554q8\" (UID: \"3bf82596-42ad-45e4-b53d-352af8564b3b\") " pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:19.084796 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:19.084758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:19.253330 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:19.253300 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8"] Jul 13 19:03:19.255495 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:03:19.255470 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bf82596_42ad_45e4_b53d_352af8564b3b.slice/crio-7f1d2586cd957bcdcca9849c80865926cff84309e246351cb588858915542408 WatchSource:0}: Error finding container 7f1d2586cd957bcdcca9849c80865926cff84309e246351cb588858915542408: Status 404 returned error can't find the container with id 7f1d2586cd957bcdcca9849c80865926cff84309e246351cb588858915542408 Jul 13 19:03:19.257849 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:19.257828 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:03:20.002906 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.002872 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" event={"ID":"3bf82596-42ad-45e4-b53d-352af8564b3b","Type":"ContainerStarted","Data":"7f1d2586cd957bcdcca9849c80865926cff84309e246351cb588858915542408"} Jul 13 19:03:20.096265 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.096223 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g"] Jul 13 19:03:20.099144 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.099112 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.102443 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.102387 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q7qnq\"" Jul 13 19:03:20.102624 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.102608 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 13 19:03:20.102835 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.102816 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 13 19:03:20.103263 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.103227 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 13 19:03:20.103660 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.103645 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 13 19:03:20.172596 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.172558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6fe140-c81e-4182-bc5b-951f92c12708-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.172596 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.172603 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l97m\" (UniqueName: \"kubernetes.io/projected/3f6fe140-c81e-4182-bc5b-951f92c12708-kube-api-access-7l97m\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.172833 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.172625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3f6fe140-c81e-4182-bc5b-951f92c12708-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.172833 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.172701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f6fe140-c81e-4182-bc5b-951f92c12708-tmp\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.172833 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.172759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3f6fe140-c81e-4182-bc5b-951f92c12708-manifests\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.194346 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.194312 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g"] Jul 13 19:03:20.273929 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.273830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l97m\" (UniqueName: \"kubernetes.io/projected/3f6fe140-c81e-4182-bc5b-951f92c12708-kube-api-access-7l97m\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.273929 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.273874 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3f6fe140-c81e-4182-bc5b-951f92c12708-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.273929 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.273895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f6fe140-c81e-4182-bc5b-951f92c12708-tmp\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.274192 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.274163 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3f6fe140-c81e-4182-bc5b-951f92c12708-manifests\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.274316 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.274301 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6fe140-c81e-4182-bc5b-951f92c12708-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.274507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.274490 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3f6fe140-c81e-4182-bc5b-951f92c12708-manifests\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.276266 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.276241 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f6fe140-c81e-4182-bc5b-951f92c12708-tmp\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.276608 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.276585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f6fe140-c81e-4182-bc5b-951f92c12708-webhook-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.276690 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.276585 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3f6fe140-c81e-4182-bc5b-951f92c12708-apiservice-cert\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.282724 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.282660 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l97m\" (UniqueName: \"kubernetes.io/projected/3f6fe140-c81e-4182-bc5b-951f92c12708-kube-api-access-7l97m\") pod \"opendatahub-operator-controller-manager-559f5498f-tdc7g\" (UID: \"3f6fe140-c81e-4182-bc5b-951f92c12708\") " pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.410387 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.410352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:20.554336 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:20.554309 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g"] Jul 13 19:03:20.556284 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:03:20.556258 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6fe140_c81e_4182_bc5b_951f92c12708.slice/crio-fb4612dfc1a96e53a196228549f75a0caa106e8da842c0fccb68f52e1013744d WatchSource:0}: Error finding container fb4612dfc1a96e53a196228549f75a0caa106e8da842c0fccb68f52e1013744d: Status 404 returned error can't find the container with id fb4612dfc1a96e53a196228549f75a0caa106e8da842c0fccb68f52e1013744d Jul 13 19:03:21.014263 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:21.014221 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" event={"ID":"3f6fe140-c81e-4182-bc5b-951f92c12708","Type":"ContainerStarted","Data":"fb4612dfc1a96e53a196228549f75a0caa106e8da842c0fccb68f52e1013744d"} Jul 13 19:03:25.038721 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:25.038666 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" event={"ID":"3bf82596-42ad-45e4-b53d-352af8564b3b","Type":"ContainerStarted","Data":"bad7a2c2905d91cd64006478853a537881772c7895651d3379d2103f07a227fd"} Jul 13 19:03:25.039153 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:25.038865 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:25.040395 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:25.040366 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" event={"ID":"3f6fe140-c81e-4182-bc5b-951f92c12708","Type":"ContainerStarted","Data":"245692b1993df7c5f5ae3cd2ac29ee52c3fb63e15e3c4e8199c8c5d7ce3397b8"} Jul 13 19:03:25.056602 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:25.056536 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" podStartSLOduration=2.083094046 podStartE2EDuration="7.056516797s" totalImagesPullingTime="4.973422751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:25.05461992 +0000 UTC m=+309.083178268" watchObservedRunningTime="2026-07-13 19:03:25.056516797 +0000 UTC m=+309.085075134" Jul 13 19:03:26.046746 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:26.046705 2580 generic.go:356] "Generic (PLEG): container finished" podID="3f6fe140-c81e-4182-bc5b-951f92c12708" containerID="245692b1993df7c5f5ae3cd2ac29ee52c3fb63e15e3c4e8199c8c5d7ce3397b8" exitCode=0 Jul 13 19:03:26.047133 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:26.046792 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" event={"ID":"3f6fe140-c81e-4182-bc5b-951f92c12708","Type":"ContainerDied","Data":"245692b1993df7c5f5ae3cd2ac29ee52c3fb63e15e3c4e8199c8c5d7ce3397b8"} Jul 13 19:03:27.053187 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:27.053146 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" event={"ID":"3f6fe140-c81e-4182-bc5b-951f92c12708","Type":"ContainerStarted","Data":"3e6f799413977161faad64c2d20c39ac9ea1bd84cef6931f0b3375671abce26e"} Jul 13 19:03:27.053563 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:27.053265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:27.082685 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:27.082604 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" podStartSLOduration=1.951155667 podStartE2EDuration="7.082586788s" totalImagesPullingTime="4.029710759s" totalInitContainerRuntime="1.101720362s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:27.080750554 +0000 UTC m=+311.109308892" watchObservedRunningTime="2026-07-13 19:03:27.082586788 +0000 UTC m=+311.111145125" Jul 13 19:03:36.049519 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:36.049487 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64bdc88f66-554q8" Jul 13 19:03:38.060664 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:38.060629 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-559f5498f-tdc7g" Jul 13 19:03:53.911943 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.911900 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2"] Jul 13 19:03:53.917942 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.917914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.923784 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.923752 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 13 19:03:53.924103 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.924084 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-kzc7w\"" Jul 13 19:03:53.933430 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.933394 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2"] Jul 13 19:03:53.963159 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963203 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963221 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhp89\" (UniqueName: \"kubernetes.io/projected/72da3b81-c8c0-4f08-acd9-90a5b9acee25-kube-api-access-xhp89\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963329 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:53.963659 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:53.963383 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064201 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064201 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064203 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064473 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhp89\" (UniqueName: \"kubernetes.io/projected/72da3b81-c8c0-4f08-acd9-90a5b9acee25-kube-api-access-xhp89\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064473 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064473 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064473 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064473 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064782 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064782 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064782 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064621 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064782 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.064977 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064903 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.065019 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.064982 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.065237 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.065218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.066986 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.066953 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.067230 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.067205 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.075256 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.075224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72da3b81-c8c0-4f08-acd9-90a5b9acee25-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.087937 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.087910 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhp89\" (UniqueName: \"kubernetes.io/projected/72da3b81-c8c0-4f08-acd9-90a5b9acee25-kube-api-access-xhp89\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4t2sl2\" (UID: \"72da3b81-c8c0-4f08-acd9-90a5b9acee25\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.228822 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.228733 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:54.380423 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:54.380391 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2"] Jul 13 19:03:54.381817 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:03:54.381788 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72da3b81_c8c0_4f08_acd9_90a5b9acee25.slice/crio-b3ca664812adb50535352958b7476db938339bb40a8fcb37472ded4d3d61d3cc WatchSource:0}: Error finding container b3ca664812adb50535352958b7476db938339bb40a8fcb37472ded4d3d61d3cc: Status 404 returned error can't find the container with id b3ca664812adb50535352958b7476db938339bb40a8fcb37472ded4d3d61d3cc Jul 13 19:03:55.192727 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:55.192662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" event={"ID":"72da3b81-c8c0-4f08-acd9-90a5b9acee25","Type":"ContainerStarted","Data":"b3ca664812adb50535352958b7476db938339bb40a8fcb37472ded4d3d61d3cc"} Jul 13 19:03:56.838662 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:56.838609 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 19:03:56.839055 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:56.838717 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 19:03:56.839055 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:56.838767 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 19:03:57.190661 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.190572 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456"] Jul 13 19:03:57.193920 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.193899 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.196311 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.196285 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 13 19:03:57.196943 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.196923 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-wfm9v\"" Jul 13 19:03:57.197025 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.196974 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 13 19:03:57.202189 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.202167 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 13 19:03:57.205588 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.205559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" event={"ID":"72da3b81-c8c0-4f08-acd9-90a5b9acee25","Type":"ContainerStarted","Data":"b5d964ad2466e0d04f475b7295194e61cf48f29200afc92e07af385ff0d8edcb"} Jul 13 19:03:57.215637 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.215606 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456"] Jul 13 19:03:57.229274 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.229231 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:57.230592 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.230560 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" start-of-body= Jul 13 19:03:57.230772 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.230629 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" podUID="72da3b81-c8c0-4f08-acd9-90a5b9acee25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.32:15021/healthz/ready\": dial tcp 10.132.0.32:15021: connect: connection refused" Jul 13 19:03:57.262191 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.262102 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" podStartSLOduration=1.808097335 podStartE2EDuration="4.262082971s" totalImagesPullingTime="2.453985636s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:03:57.261046975 +0000 UTC m=+341.289605336" watchObservedRunningTime="2026-07-13 19:03:57.262082971 +0000 UTC m=+341.290641308" Jul 13 19:03:57.299433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299394 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30480d3-cadf-4dda-ba68-69b131536262-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.299587 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d30480d3-cadf-4dda-ba68-69b131536262-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.299587 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30480d3-cadf-4dda-ba68-69b131536262-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.299587 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.299587 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.299836 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299612 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc9ql\" (UniqueName: \"kubernetes.io/projected/d30480d3-cadf-4dda-ba68-69b131536262-kube-api-access-mc9ql\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.299836 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.299797 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.400835 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.400791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.400845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30480d3-cadf-4dda-ba68-69b131536262-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.400877 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d30480d3-cadf-4dda-ba68-69b131536262-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.400909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30480d3-cadf-4dda-ba68-69b131536262-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.400946 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.400979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.401005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9ql\" (UniqueName: \"kubernetes.io/projected/d30480d3-cadf-4dda-ba68-69b131536262-kube-api-access-mc9ql\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.401813 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.401785 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d30480d3-cadf-4dda-ba68-69b131536262-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.403189 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.403155 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d30480d3-cadf-4dda-ba68-69b131536262-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.403359 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.403337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.403644 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.403625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.403748 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.403663 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d30480d3-cadf-4dda-ba68-69b131536262-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.414189 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.414155 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc9ql\" (UniqueName: \"kubernetes.io/projected/d30480d3-cadf-4dda-ba68-69b131536262-kube-api-access-mc9ql\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.422099 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.422072 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d30480d3-cadf-4dda-ba68-69b131536262-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-p9456\" (UID: \"d30480d3-cadf-4dda-ba68-69b131536262\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.504781 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.504414 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:03:57.673635 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:57.673607 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456"] Jul 13 19:03:57.676925 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:03:57.676895 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd30480d3_cadf_4dda_ba68_69b131536262.slice/crio-d993035b538be6caac69e26161010d5f95f123f465b8508b2f7a7f7dc5ef2316 WatchSource:0}: Error finding container d993035b538be6caac69e26161010d5f95f123f465b8508b2f7a7f7dc5ef2316: Status 404 returned error can't find the container with id d993035b538be6caac69e26161010d5f95f123f465b8508b2f7a7f7dc5ef2316 Jul 13 19:03:58.212898 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:58.212854 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" event={"ID":"d30480d3-cadf-4dda-ba68-69b131536262","Type":"ContainerStarted","Data":"d993035b538be6caac69e26161010d5f95f123f465b8508b2f7a7f7dc5ef2316"} Jul 13 19:03:58.233227 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:58.233196 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:59.218339 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:59.218306 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:03:59.219502 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:03:59.219483 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4t2sl2" Jul 13 19:04:03.123863 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:03.123815 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 19:04:03.124183 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:03.123893 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 13 19:04:03.241149 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:03.241027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" event={"ID":"d30480d3-cadf-4dda-ba68-69b131536262","Type":"ContainerStarted","Data":"f5865d7679d9634480ce672bf67c3f9fd5b346611a3f861c0a8ff90162fdd09f"} Jul 13 19:04:03.241149 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:03.241109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:04:03.278616 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:03.278563 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" podStartSLOduration=0.834122746 podStartE2EDuration="6.278544751s" totalImagesPullingTime="5.444422005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:04:03.277345402 +0000 UTC m=+347.305903751" watchObservedRunningTime="2026-07-13 19:04:03.278544751 +0000 UTC m=+347.307103090" Jul 13 19:04:04.248120 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:04.248084 2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-p9456 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 13 19:04:04.248514 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:04.248152 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" podUID="d30480d3-cadf-4dda-ba68-69b131536262" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 19:04:07.248087 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:07.248053 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-p9456" Jul 13 19:04:57.698169 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.698134 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf"] Jul 13 19:04:57.700847 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.700820 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.703574 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.703536 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 13 19:04:57.703997 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.703976 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lsvdw\"" Jul 13 19:04:57.704559 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.704538 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 13 19:04:57.718131 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.718098 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf"] Jul 13 19:04:57.851351 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.851307 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.851556 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.851369 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5scmz\" (UniqueName: \"kubernetes.io/projected/3fea2deb-622b-49d6-b8a0-8fd77608337e-kube-api-access-5scmz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.851556 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.851405 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.952964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.952870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.952964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.952929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5scmz\" (UniqueName: \"kubernetes.io/projected/3fea2deb-622b-49d6-b8a0-8fd77608337e-kube-api-access-5scmz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.952964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.952960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.953368 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.953352 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.953412 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.953353 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:57.962521 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:57.962494 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5scmz\" (UniqueName: \"kubernetes.io/projected/3fea2deb-622b-49d6-b8a0-8fd77608337e-kube-api-access-5scmz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:58.011896 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.011849 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:04:58.093909 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.093877 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z"] Jul 13 19:04:58.096446 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.096423 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.107529 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.107426 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z"] Jul 13 19:04:58.155086 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.155036 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.155086 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.155088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.155330 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.155180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvrbg\" (UniqueName: \"kubernetes.io/projected/71154e7c-f067-4ba4-912f-803d6dbe83b4-kube-api-access-nvrbg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.161361 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.161331 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf"] Jul 13 19:04:58.163319 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:04:58.163291 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fea2deb_622b_49d6_b8a0_8fd77608337e.slice/crio-cafa6d2810db3996b0349449e14b1594517d48e307dc04cfbf6bfe1ea0c3a7f7 WatchSource:0}: Error finding container cafa6d2810db3996b0349449e14b1594517d48e307dc04cfbf6bfe1ea0c3a7f7: Status 404 returned error can't find the container with id cafa6d2810db3996b0349449e14b1594517d48e307dc04cfbf6bfe1ea0c3a7f7 Jul 13 19:04:58.256146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.256100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvrbg\" (UniqueName: \"kubernetes.io/projected/71154e7c-f067-4ba4-912f-803d6dbe83b4-kube-api-access-nvrbg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.256347 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.256178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.256347 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.256202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.256607 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.256586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.256697 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.256613 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.268467 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.268438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvrbg\" (UniqueName: \"kubernetes.io/projected/71154e7c-f067-4ba4-912f-803d6dbe83b4-kube-api-access-nvrbg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.409967 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.409920 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:04:58.507423 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.507382 2580 generic.go:356] "Generic (PLEG): container finished" podID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerID="704972814d894ed3e9e4f496fa44ad179f485c0b546e61fa8703f34e28497dac" exitCode=0 Jul 13 19:04:58.507627 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.507499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" event={"ID":"3fea2deb-622b-49d6-b8a0-8fd77608337e","Type":"ContainerDied","Data":"704972814d894ed3e9e4f496fa44ad179f485c0b546e61fa8703f34e28497dac"} Jul 13 19:04:58.507627 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.507569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" event={"ID":"3fea2deb-622b-49d6-b8a0-8fd77608337e","Type":"ContainerStarted","Data":"cafa6d2810db3996b0349449e14b1594517d48e307dc04cfbf6bfe1ea0c3a7f7"} Jul 13 19:04:58.521125 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.521090 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp"] Jul 13 19:04:58.528016 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.527982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.549155 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.549118 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp"] Jul 13 19:04:58.582691 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.582632 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z"] Jul 13 19:04:58.586115 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:04:58.586083 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71154e7c_f067_4ba4_912f_803d6dbe83b4.slice/crio-6bec6fdccf0790de2158492b7427e389140f20291fcabf0cacaecd75109f8df7 WatchSource:0}: Error finding container 6bec6fdccf0790de2158492b7427e389140f20291fcabf0cacaecd75109f8df7: Status 404 returned error can't find the container with id 6bec6fdccf0790de2158492b7427e389140f20291fcabf0cacaecd75109f8df7 Jul 13 19:04:58.658914 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.658872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.659085 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.658964 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.659085 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.659040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhv8\" (UniqueName: \"kubernetes.io/projected/31ed856b-2822-4a8a-ba4a-5347c838c9ec-kube-api-access-xjhv8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.759508 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.759416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhv8\" (UniqueName: \"kubernetes.io/projected/31ed856b-2822-4a8a-ba4a-5347c838c9ec-kube-api-access-xjhv8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.759508 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.759468 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.759970 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.759522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.759970 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.759948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.760038 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.759959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.769076 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.769050 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhv8\" (UniqueName: \"kubernetes.io/projected/31ed856b-2822-4a8a-ba4a-5347c838c9ec-kube-api-access-xjhv8\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.837839 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.837797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:04:58.985223 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:58.984050 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp"] Jul 13 19:04:58.989090 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:04:58.989040 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ed856b_2822_4a8a_ba4a_5347c838c9ec.slice/crio-d4d56b5a017d68f2cb5c28189e0411afa9ba528c24a630015bac45380831dee2 WatchSource:0}: Error finding container d4d56b5a017d68f2cb5c28189e0411afa9ba528c24a630015bac45380831dee2: Status 404 returned error can't find the container with id d4d56b5a017d68f2cb5c28189e0411afa9ba528c24a630015bac45380831dee2 Jul 13 19:04:59.099514 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.099486 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw"] Jul 13 19:04:59.102272 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.102255 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.118143 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.118108 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw"] Jul 13 19:04:59.162588 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.162553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.162769 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.162610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.162769 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.162712 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhtd\" (UniqueName: \"kubernetes.io/projected/84af5016-3ed7-4857-a098-09f1efa27982-kube-api-access-8dhtd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.264170 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.264127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhtd\" (UniqueName: \"kubernetes.io/projected/84af5016-3ed7-4857-a098-09f1efa27982-kube-api-access-8dhtd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.264357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.264223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.264357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.264269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.264744 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.264721 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.264744 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.264738 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.274477 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.274404 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhtd\" (UniqueName: \"kubernetes.io/projected/84af5016-3ed7-4857-a098-09f1efa27982-kube-api-access-8dhtd\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.460636 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.460590 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:04:59.517298 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.517262 2580 generic.go:356] "Generic (PLEG): container finished" podID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerID="9e6cc619173b9072cad2a0c370440984cec82f3638d6ba832fae2fdc5d8ba829" exitCode=0 Jul 13 19:04:59.517502 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.517349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" event={"ID":"31ed856b-2822-4a8a-ba4a-5347c838c9ec","Type":"ContainerDied","Data":"9e6cc619173b9072cad2a0c370440984cec82f3638d6ba832fae2fdc5d8ba829"} Jul 13 19:04:59.517502 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.517441 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" event={"ID":"31ed856b-2822-4a8a-ba4a-5347c838c9ec","Type":"ContainerStarted","Data":"d4d56b5a017d68f2cb5c28189e0411afa9ba528c24a630015bac45380831dee2"} Jul 13 19:04:59.522416 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.522388 2580 generic.go:356] "Generic (PLEG): container finished" podID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerID="d5101aba9dc6f9f87f0c44935042560c316d68ac625c9d2ebe5f2752c50e4ba8" exitCode=0 Jul 13 19:04:59.522624 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.522471 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" event={"ID":"71154e7c-f067-4ba4-912f-803d6dbe83b4","Type":"ContainerDied","Data":"d5101aba9dc6f9f87f0c44935042560c316d68ac625c9d2ebe5f2752c50e4ba8"} Jul 13 19:04:59.522624 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.522529 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" event={"ID":"71154e7c-f067-4ba4-912f-803d6dbe83b4","Type":"ContainerStarted","Data":"6bec6fdccf0790de2158492b7427e389140f20291fcabf0cacaecd75109f8df7"} Jul 13 19:04:59.608284 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:04:59.608254 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw"] Jul 13 19:04:59.609632 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:04:59.609608 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84af5016_3ed7_4857_a098_09f1efa27982.slice/crio-2ce5ec6e4147af8e55d417fd1ed423d9153e3a7414e97598af39263dd68f7660 WatchSource:0}: Error finding container 2ce5ec6e4147af8e55d417fd1ed423d9153e3a7414e97598af39263dd68f7660: Status 404 returned error can't find the container with id 2ce5ec6e4147af8e55d417fd1ed423d9153e3a7414e97598af39263dd68f7660 Jul 13 19:05:00.529954 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:00.529820 2580 generic.go:356] "Generic (PLEG): container finished" podID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerID="b13bf6054277623aca4782d2eece91bf8ba1897e700d7e7f3b7e83c07b85bb1f" exitCode=0 Jul 13 19:05:00.529954 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:00.529926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" event={"ID":"3fea2deb-622b-49d6-b8a0-8fd77608337e","Type":"ContainerDied","Data":"b13bf6054277623aca4782d2eece91bf8ba1897e700d7e7f3b7e83c07b85bb1f"} Jul 13 19:05:00.531821 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:00.531788 2580 generic.go:356] "Generic (PLEG): container finished" podID="84af5016-3ed7-4857-a098-09f1efa27982" containerID="48fe620ba446e62b7a8d9aa76f47eb9ca3325a76bf49e9bc93f867ab02bd5561" exitCode=0 Jul 13 19:05:00.532017 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:00.531971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" event={"ID":"84af5016-3ed7-4857-a098-09f1efa27982","Type":"ContainerDied","Data":"48fe620ba446e62b7a8d9aa76f47eb9ca3325a76bf49e9bc93f867ab02bd5561"} Jul 13 19:05:00.532167 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:00.532032 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" event={"ID":"84af5016-3ed7-4857-a098-09f1efa27982","Type":"ContainerStarted","Data":"2ce5ec6e4147af8e55d417fd1ed423d9153e3a7414e97598af39263dd68f7660"} Jul 13 19:05:00.534315 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:00.534283 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" event={"ID":"71154e7c-f067-4ba4-912f-803d6dbe83b4","Type":"ContainerStarted","Data":"6dfee6fcb98aedb805c679dff910dcb470ce9f6ff9aa0cfd78436647ed356a09"} Jul 13 19:05:01.544801 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:01.544766 2580 generic.go:356] "Generic (PLEG): container finished" podID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerID="6dfee6fcb98aedb805c679dff910dcb470ce9f6ff9aa0cfd78436647ed356a09" exitCode=0 Jul 13 19:05:01.545305 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:01.544848 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" event={"ID":"71154e7c-f067-4ba4-912f-803d6dbe83b4","Type":"ContainerDied","Data":"6dfee6fcb98aedb805c679dff910dcb470ce9f6ff9aa0cfd78436647ed356a09"} Jul 13 19:05:01.547240 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:01.547216 2580 generic.go:356] "Generic (PLEG): container finished" podID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerID="8848415880f85c40a81d3f4c073bce9df14a21dfbd6b4526c8be5991a8c478a0" exitCode=0 Jul 13 19:05:01.547364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:01.547289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" event={"ID":"3fea2deb-622b-49d6-b8a0-8fd77608337e","Type":"ContainerDied","Data":"8848415880f85c40a81d3f4c073bce9df14a21dfbd6b4526c8be5991a8c478a0"} Jul 13 19:05:01.549145 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:01.549121 2580 generic.go:356] "Generic (PLEG): container finished" podID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerID="8d3115b82d0064d0a980f10ee388104beb3c9fd68ea60928af8335d3fa5d2473" exitCode=0 Jul 13 19:05:01.549276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:01.549149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" event={"ID":"31ed856b-2822-4a8a-ba4a-5347c838c9ec","Type":"ContainerDied","Data":"8d3115b82d0064d0a980f10ee388104beb3c9fd68ea60928af8335d3fa5d2473"} Jul 13 19:05:02.555238 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.555203 2580 generic.go:356] "Generic (PLEG): container finished" podID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerID="7da11518b1280a5c962eb862405b038dd10323388d6f7e99604a35bd5efa4126" exitCode=0 Jul 13 19:05:02.555642 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.555286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" event={"ID":"31ed856b-2822-4a8a-ba4a-5347c838c9ec","Type":"ContainerDied","Data":"7da11518b1280a5c962eb862405b038dd10323388d6f7e99604a35bd5efa4126"} Jul 13 19:05:02.557302 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.557279 2580 generic.go:356] "Generic (PLEG): container finished" podID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerID="759e2a30f688acb3ebd882dc50fbeb369a80bd1862e53366fb377c4f6b47ac21" exitCode=0 Jul 13 19:05:02.557392 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.557370 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" event={"ID":"71154e7c-f067-4ba4-912f-803d6dbe83b4","Type":"ContainerDied","Data":"759e2a30f688acb3ebd882dc50fbeb369a80bd1862e53366fb377c4f6b47ac21"} Jul 13 19:05:02.698903 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.698869 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:05:02.796245 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.796209 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5scmz\" (UniqueName: \"kubernetes.io/projected/3fea2deb-622b-49d6-b8a0-8fd77608337e-kube-api-access-5scmz\") pod \"3fea2deb-622b-49d6-b8a0-8fd77608337e\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " Jul 13 19:05:02.796503 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.796387 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-bundle\") pod \"3fea2deb-622b-49d6-b8a0-8fd77608337e\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " Jul 13 19:05:02.796503 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.796438 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-util\") pod \"3fea2deb-622b-49d6-b8a0-8fd77608337e\" (UID: \"3fea2deb-622b-49d6-b8a0-8fd77608337e\") " Jul 13 19:05:02.796890 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.796866 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-bundle" (OuterVolumeSpecName: "bundle") pod "3fea2deb-622b-49d6-b8a0-8fd77608337e" (UID: "3fea2deb-622b-49d6-b8a0-8fd77608337e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:02.798520 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.798497 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3fea2deb-622b-49d6-b8a0-8fd77608337e-kube-api-access-5scmz" (OuterVolumeSpecName: "kube-api-access-5scmz") pod "3fea2deb-622b-49d6-b8a0-8fd77608337e" (UID: "3fea2deb-622b-49d6-b8a0-8fd77608337e"). InnerVolumeSpecName "kube-api-access-5scmz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:05:02.801874 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.801826 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-util" (OuterVolumeSpecName: "util") pod "3fea2deb-622b-49d6-b8a0-8fd77608337e" (UID: "3fea2deb-622b-49d6-b8a0-8fd77608337e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:02.897490 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.897451 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:02.897490 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.897491 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5scmz\" (UniqueName: \"kubernetes.io/projected/3fea2deb-622b-49d6-b8a0-8fd77608337e-kube-api-access-5scmz\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:02.897743 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:02.897507 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3fea2deb-622b-49d6-b8a0-8fd77608337e-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:03.564509 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.564472 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" event={"ID":"3fea2deb-622b-49d6-b8a0-8fd77608337e","Type":"ContainerDied","Data":"cafa6d2810db3996b0349449e14b1594517d48e307dc04cfbf6bfe1ea0c3a7f7"} Jul 13 19:05:03.564939 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.564523 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cafa6d2810db3996b0349449e14b1594517d48e307dc04cfbf6bfe1ea0c3a7f7" Jul 13 19:05:03.564939 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.564483 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf" Jul 13 19:05:03.566357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.566332 2580 generic.go:356] "Generic (PLEG): container finished" podID="84af5016-3ed7-4857-a098-09f1efa27982" containerID="630bc45893f64e4e801f8485829af42ceb17192724db5701f6407bdfe0b19da5" exitCode=0 Jul 13 19:05:03.566476 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.566423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" event={"ID":"84af5016-3ed7-4857-a098-09f1efa27982","Type":"ContainerDied","Data":"630bc45893f64e4e801f8485829af42ceb17192724db5701f6407bdfe0b19da5"} Jul 13 19:05:03.717492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.717463 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:05:03.784327 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.784295 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:05:03.807220 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.807186 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-util\") pod \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " Jul 13 19:05:03.807410 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.807254 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjhv8\" (UniqueName: \"kubernetes.io/projected/31ed856b-2822-4a8a-ba4a-5347c838c9ec-kube-api-access-xjhv8\") pod \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " Jul 13 19:05:03.807410 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.807313 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-bundle\") pod \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\" (UID: \"31ed856b-2822-4a8a-ba4a-5347c838c9ec\") " Jul 13 19:05:03.808076 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.808045 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-bundle" (OuterVolumeSpecName: "bundle") pod "31ed856b-2822-4a8a-ba4a-5347c838c9ec" (UID: "31ed856b-2822-4a8a-ba4a-5347c838c9ec"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:03.809558 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.809528 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ed856b-2822-4a8a-ba4a-5347c838c9ec-kube-api-access-xjhv8" (OuterVolumeSpecName: "kube-api-access-xjhv8") pod "31ed856b-2822-4a8a-ba4a-5347c838c9ec" (UID: "31ed856b-2822-4a8a-ba4a-5347c838c9ec"). InnerVolumeSpecName "kube-api-access-xjhv8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:05:03.812608 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.812583 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-util" (OuterVolumeSpecName: "util") pod "31ed856b-2822-4a8a-ba4a-5347c838c9ec" (UID: "31ed856b-2822-4a8a-ba4a-5347c838c9ec"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:03.908484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.908432 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-bundle\") pod \"71154e7c-f067-4ba4-912f-803d6dbe83b4\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " Jul 13 19:05:03.908701 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.908510 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvrbg\" (UniqueName: \"kubernetes.io/projected/71154e7c-f067-4ba4-912f-803d6dbe83b4-kube-api-access-nvrbg\") pod \"71154e7c-f067-4ba4-912f-803d6dbe83b4\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " Jul 13 19:05:03.908701 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.908537 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-util\") pod \"71154e7c-f067-4ba4-912f-803d6dbe83b4\" (UID: \"71154e7c-f067-4ba4-912f-803d6dbe83b4\") " Jul 13 19:05:03.908896 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.908868 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:03.908896 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.908894 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjhv8\" (UniqueName: \"kubernetes.io/projected/31ed856b-2822-4a8a-ba4a-5347c838c9ec-kube-api-access-xjhv8\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:03.909043 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.908908 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/31ed856b-2822-4a8a-ba4a-5347c838c9ec-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:03.909085 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.909042 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-bundle" (OuterVolumeSpecName: "bundle") pod "71154e7c-f067-4ba4-912f-803d6dbe83b4" (UID: "71154e7c-f067-4ba4-912f-803d6dbe83b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:03.910642 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.910622 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71154e7c-f067-4ba4-912f-803d6dbe83b4-kube-api-access-nvrbg" (OuterVolumeSpecName: "kube-api-access-nvrbg") pod "71154e7c-f067-4ba4-912f-803d6dbe83b4" (UID: "71154e7c-f067-4ba4-912f-803d6dbe83b4"). InnerVolumeSpecName "kube-api-access-nvrbg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:05:03.914637 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:03.914601 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-util" (OuterVolumeSpecName: "util") pod "71154e7c-f067-4ba4-912f-803d6dbe83b4" (UID: "71154e7c-f067-4ba4-912f-803d6dbe83b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:04.009480 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.009443 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvrbg\" (UniqueName: \"kubernetes.io/projected/71154e7c-f067-4ba4-912f-803d6dbe83b4-kube-api-access-nvrbg\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:04.009480 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.009475 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:04.009480 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.009488 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/71154e7c-f067-4ba4-912f-803d6dbe83b4-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:04.574134 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.574104 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" Jul 13 19:05:04.574521 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.574149 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z" event={"ID":"71154e7c-f067-4ba4-912f-803d6dbe83b4","Type":"ContainerDied","Data":"6bec6fdccf0790de2158492b7427e389140f20291fcabf0cacaecd75109f8df7"} Jul 13 19:05:04.574521 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.574193 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bec6fdccf0790de2158492b7427e389140f20291fcabf0cacaecd75109f8df7" Jul 13 19:05:04.576345 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.576305 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" Jul 13 19:05:04.576488 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.576302 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp" event={"ID":"31ed856b-2822-4a8a-ba4a-5347c838c9ec","Type":"ContainerDied","Data":"d4d56b5a017d68f2cb5c28189e0411afa9ba528c24a630015bac45380831dee2"} Jul 13 19:05:04.576488 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.576445 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4d56b5a017d68f2cb5c28189e0411afa9ba528c24a630015bac45380831dee2" Jul 13 19:05:04.578755 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.578729 2580 generic.go:356] "Generic (PLEG): container finished" podID="84af5016-3ed7-4857-a098-09f1efa27982" containerID="413061f9d56fb21164d77ffafe75d2c4d3e6ed9d4e4cbb772736e8244027db56" exitCode=0 Jul 13 19:05:04.578874 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:04.578788 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" event={"ID":"84af5016-3ed7-4857-a098-09f1efa27982","Type":"ContainerDied","Data":"413061f9d56fb21164d77ffafe75d2c4d3e6ed9d4e4cbb772736e8244027db56"} Jul 13 19:05:05.722108 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.722061 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:05:05.826509 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.826474 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-bundle\") pod \"84af5016-3ed7-4857-a098-09f1efa27982\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " Jul 13 19:05:05.826665 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.826553 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-util\") pod \"84af5016-3ed7-4857-a098-09f1efa27982\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " Jul 13 19:05:05.826665 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.826618 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dhtd\" (UniqueName: \"kubernetes.io/projected/84af5016-3ed7-4857-a098-09f1efa27982-kube-api-access-8dhtd\") pod \"84af5016-3ed7-4857-a098-09f1efa27982\" (UID: \"84af5016-3ed7-4857-a098-09f1efa27982\") " Jul 13 19:05:05.827146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.827113 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-bundle" (OuterVolumeSpecName: "bundle") pod "84af5016-3ed7-4857-a098-09f1efa27982" (UID: "84af5016-3ed7-4857-a098-09f1efa27982"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:05.828847 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.828821 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84af5016-3ed7-4857-a098-09f1efa27982-kube-api-access-8dhtd" (OuterVolumeSpecName: "kube-api-access-8dhtd") pod "84af5016-3ed7-4857-a098-09f1efa27982" (UID: "84af5016-3ed7-4857-a098-09f1efa27982"). InnerVolumeSpecName "kube-api-access-8dhtd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:05:05.832947 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.832910 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-util" (OuterVolumeSpecName: "util") pod "84af5016-3ed7-4857-a098-09f1efa27982" (UID: "84af5016-3ed7-4857-a098-09f1efa27982"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 19:05:05.927791 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.927744 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dhtd\" (UniqueName: \"kubernetes.io/projected/84af5016-3ed7-4857-a098-09f1efa27982-kube-api-access-8dhtd\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:05.927791 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.927781 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:05.927791 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:05.927792 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/84af5016-3ed7-4857-a098-09f1efa27982-util\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:06.591549 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:06.591521 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" Jul 13 19:05:06.591549 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:06.591540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw" event={"ID":"84af5016-3ed7-4857-a098-09f1efa27982","Type":"ContainerDied","Data":"2ce5ec6e4147af8e55d417fd1ed423d9153e3a7414e97598af39263dd68f7660"} Jul 13 19:05:06.591790 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:06.591589 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ce5ec6e4147af8e55d417fd1ed423d9153e3a7414e97598af39263dd68f7660" Jul 13 19:05:23.225074 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:23.225033 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68cdd6b489-f5fdr"] Jul 13 19:05:48.245870 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.245771 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68cdd6b489-f5fdr" podUID="bfadfe9d-ace3-40fe-a138-818ae8e638f3" containerName="console" containerID="cri-o://9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec" gracePeriod=15 Jul 13 19:05:48.507248 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.507172 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:05:48.592830 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.592798 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-oauth-serving-cert\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.592830 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.592840 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-service-ca\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.593096 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.592857 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-config\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.593096 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.592881 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-trusted-ca-bundle\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.593096 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.592913 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7v4n\" (UniqueName: \"kubernetes.io/projected/bfadfe9d-ace3-40fe-a138-818ae8e638f3-kube-api-access-z7v4n\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.593096 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.592941 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-serving-cert\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.593096 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.593006 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-oauth-config\") pod \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\" (UID: \"bfadfe9d-ace3-40fe-a138-818ae8e638f3\") " Jul 13 19:05:48.593340 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.593306 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-service-ca" (OuterVolumeSpecName: "service-ca") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:05:48.593396 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.593359 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-config" (OuterVolumeSpecName: "console-config") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:05:48.593396 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.593351 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:05:48.593783 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.593745 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:05:48.595425 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.595397 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:05:48.595514 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.595457 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:05:48.595514 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.595496 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfadfe9d-ace3-40fe-a138-818ae8e638f3-kube-api-access-z7v4n" (OuterVolumeSpecName: "kube-api-access-z7v4n") pod "bfadfe9d-ace3-40fe-a138-818ae8e638f3" (UID: "bfadfe9d-ace3-40fe-a138-818ae8e638f3"). InnerVolumeSpecName "kube-api-access-z7v4n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:05:48.694116 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694080 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-oauth-config\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.694116 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694113 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-oauth-serving-cert\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.694116 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694123 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.694349 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694132 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-config\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.694349 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694140 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfadfe9d-ace3-40fe-a138-818ae8e638f3-trusted-ca-bundle\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.694349 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694149 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7v4n\" (UniqueName: \"kubernetes.io/projected/bfadfe9d-ace3-40fe-a138-818ae8e638f3-kube-api-access-z7v4n\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.694349 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.694161 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfadfe9d-ace3-40fe-a138-818ae8e638f3-console-serving-cert\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:05:48.815030 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.814943 2580 generic.go:356] "Generic (PLEG): container finished" podID="bfadfe9d-ace3-40fe-a138-818ae8e638f3" containerID="9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec" exitCode=0 Jul 13 19:05:48.815030 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.814988 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68cdd6b489-f5fdr" Jul 13 19:05:48.815030 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.815003 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68cdd6b489-f5fdr" event={"ID":"bfadfe9d-ace3-40fe-a138-818ae8e638f3","Type":"ContainerDied","Data":"9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec"} Jul 13 19:05:48.815283 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.815045 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68cdd6b489-f5fdr" event={"ID":"bfadfe9d-ace3-40fe-a138-818ae8e638f3","Type":"ContainerDied","Data":"e35f46e479c38227de296380809caace50754f884962712a28910a4b23a1403e"} Jul 13 19:05:48.815283 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.815066 2580 scope.go:117] "RemoveContainer" containerID="9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec" Jul 13 19:05:48.825147 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.825124 2580 scope.go:117] "RemoveContainer" containerID="9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec" Jul 13 19:05:48.825487 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:05:48.825466 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec\": container with ID starting with 9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec not found: ID does not exist" containerID="9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec" Jul 13 19:05:48.825564 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.825501 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec"} err="failed to get container status \"9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec\": rpc error: code = NotFound desc = could not find container \"9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec\": container with ID starting with 9648203a974fcf6c4e1959b926ef1d5b6cf65d8e4b6a7f0b3c7b225bae98f7ec not found: ID does not exist" Jul 13 19:05:48.853513 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.853476 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68cdd6b489-f5fdr"] Jul 13 19:05:48.862374 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:48.862326 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68cdd6b489-f5fdr"] Jul 13 19:05:50.546376 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:05:50.546335 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfadfe9d-ace3-40fe-a138-818ae8e638f3" path="/var/lib/kubelet/pods/bfadfe9d-ace3-40fe-a138-818ae8e638f3/volumes" Jul 13 19:07:16.495922 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.495824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-54f69f5bc-pvb4r"] Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496166 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496176 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496185 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496190 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496200 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496207 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496216 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496221 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496228 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496233 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496240 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfadfe9d-ace3-40fe-a138-818ae8e638f3" containerName="console" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496245 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfadfe9d-ace3-40fe-a138-818ae8e638f3" containerName="console" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496251 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496256 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496262 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496267 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496275 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496281 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496286 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496291 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496304 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496309 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="util" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496319 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496324 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="pull" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496330 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="extract" Jul 13 19:07:16.496334 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496335 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="extract" Jul 13 19:07:16.497115 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496382 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfadfe9d-ace3-40fe-a138-818ae8e638f3" containerName="console" Jul 13 19:07:16.497115 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496388 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ed856b-2822-4a8a-ba4a-5347c838c9ec" containerName="extract" Jul 13 19:07:16.497115 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496395 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3fea2deb-622b-49d6-b8a0-8fd77608337e" containerName="extract" Jul 13 19:07:16.497115 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496400 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="84af5016-3ed7-4857-a098-09f1efa27982" containerName="extract" Jul 13 19:07:16.497115 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.496407 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="71154e7c-f067-4ba4-912f-803d6dbe83b4" containerName="extract" Jul 13 19:07:16.499370 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.499346 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.504048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.504014 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 13 19:07:16.504221 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.504030 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 13 19:07:16.504221 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.504030 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-88m76\"" Jul 13 19:07:16.514016 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.513979 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54f69f5bc-pvb4r"] Jul 13 19:07:16.562830 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.562784 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfs47\" (UniqueName: \"kubernetes.io/projected/2c17f8bb-cf44-4b07-b267-cb95fb87f277-kube-api-access-lfs47\") pod \"maas-controller-54f69f5bc-pvb4r\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.563021 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.562857 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c17f8bb-cf44-4b07-b267-cb95fb87f277-webhook-cert\") pod \"maas-controller-54f69f5bc-pvb4r\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.663357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.663323 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c17f8bb-cf44-4b07-b267-cb95fb87f277-webhook-cert\") pod \"maas-controller-54f69f5bc-pvb4r\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.663515 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.663416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfs47\" (UniqueName: \"kubernetes.io/projected/2c17f8bb-cf44-4b07-b267-cb95fb87f277-kube-api-access-lfs47\") pod \"maas-controller-54f69f5bc-pvb4r\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.665877 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.665851 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c17f8bb-cf44-4b07-b267-cb95fb87f277-webhook-cert\") pod \"maas-controller-54f69f5bc-pvb4r\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.672904 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.672872 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfs47\" (UniqueName: \"kubernetes.io/projected/2c17f8bb-cf44-4b07-b267-cb95fb87f277-kube-api-access-lfs47\") pod \"maas-controller-54f69f5bc-pvb4r\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.810905 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.810811 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:16.963385 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:16.963346 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-54f69f5bc-pvb4r"] Jul 13 19:07:16.964769 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:07:16.964737 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c17f8bb_cf44_4b07_b267_cb95fb87f277.slice/crio-a74bc62ca3283db61d26bd81d6442ec6e58c5446c2938ff4fd75afa29da93b13 WatchSource:0}: Error finding container a74bc62ca3283db61d26bd81d6442ec6e58c5446c2938ff4fd75afa29da93b13: Status 404 returned error can't find the container with id a74bc62ca3283db61d26bd81d6442ec6e58c5446c2938ff4fd75afa29da93b13 Jul 13 19:07:17.266474 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:17.266437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" event={"ID":"2c17f8bb-cf44-4b07-b267-cb95fb87f277","Type":"ContainerStarted","Data":"a74bc62ca3283db61d26bd81d6442ec6e58c5446c2938ff4fd75afa29da93b13"} Jul 13 19:07:20.285073 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:20.285040 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" event={"ID":"2c17f8bb-cf44-4b07-b267-cb95fb87f277","Type":"ContainerStarted","Data":"8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355"} Jul 13 19:07:20.285480 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:20.285110 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:20.309969 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:20.309910 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" podStartSLOduration=1.742658925 podStartE2EDuration="4.309892451s" totalImagesPullingTime="2.567233526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:07:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:07:20.307267415 +0000 UTC m=+544.335825770" watchObservedRunningTime="2026-07-13 19:07:20.309892451 +0000 UTC m=+544.338450846" Jul 13 19:07:31.292954 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:31.292922 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:07:49.810079 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.810035 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-779d67c848-9hpgg"] Jul 13 19:07:49.813444 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.813423 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:49.816156 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.816126 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 13 19:07:49.817182 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.817159 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 13 19:07:49.817365 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.817345 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 13 19:07:49.817517 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.817493 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-rpxts\"" Jul 13 19:07:49.823558 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.823533 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-779d67c848-9hpgg"] Jul 13 19:07:49.955640 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.955599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290a08de-d79d-4db1-97b3-717d202978a2-maas-api-tls\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:49.955640 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.955641 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290a08de-d79d-4db1-97b3-717d202978a2-openshift-service-ca\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:49.955899 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:49.955777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs7np\" (UniqueName: \"kubernetes.io/projected/290a08de-d79d-4db1-97b3-717d202978a2-kube-api-access-hs7np\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.056856 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.056813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290a08de-d79d-4db1-97b3-717d202978a2-maas-api-tls\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.056856 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.056858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290a08de-d79d-4db1-97b3-717d202978a2-openshift-service-ca\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.057108 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.056891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs7np\" (UniqueName: \"kubernetes.io/projected/290a08de-d79d-4db1-97b3-717d202978a2-kube-api-access-hs7np\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.057595 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.057569 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/290a08de-d79d-4db1-97b3-717d202978a2-openshift-service-ca\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.059324 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.059304 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/290a08de-d79d-4db1-97b3-717d202978a2-maas-api-tls\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.067489 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.067417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs7np\" (UniqueName: \"kubernetes.io/projected/290a08de-d79d-4db1-97b3-717d202978a2-kube-api-access-hs7np\") pod \"maas-api-779d67c848-9hpgg\" (UID: \"290a08de-d79d-4db1-97b3-717d202978a2\") " pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.125436 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.125400 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:50.301502 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:07:50.301450 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290a08de_d79d_4db1_97b3_717d202978a2.slice/crio-e0cc55cc5c6495ab5b3e73f17191f497b6099ad7e154db30689b1f63b808dd1c WatchSource:0}: Error finding container e0cc55cc5c6495ab5b3e73f17191f497b6099ad7e154db30689b1f63b808dd1c: Status 404 returned error can't find the container with id e0cc55cc5c6495ab5b3e73f17191f497b6099ad7e154db30689b1f63b808dd1c Jul 13 19:07:50.304931 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.304905 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-779d67c848-9hpgg"] Jul 13 19:07:50.451055 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.451021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" event={"ID":"290a08de-d79d-4db1-97b3-717d202978a2","Type":"ContainerStarted","Data":"e0cc55cc5c6495ab5b3e73f17191f497b6099ad7e154db30689b1f63b808dd1c"} Jul 13 19:07:50.609924 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.609887 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz"] Jul 13 19:07:50.614513 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.614488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.617190 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.617162 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 13 19:07:50.617190 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.617178 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jdrkn\"" Jul 13 19:07:50.625329 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.625299 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz"] Jul 13 19:07:50.764121 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.764019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4808c5db-8345-415c-b76e-9b2a59339775-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-sjxjz\" (UID: \"4808c5db-8345-415c-b76e-9b2a59339775\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.764121 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.764061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq8x7\" (UniqueName: \"kubernetes.io/projected/4808c5db-8345-415c-b76e-9b2a59339775-kube-api-access-tq8x7\") pod \"payload-pre-processing-74584bb9f8-sjxjz\" (UID: \"4808c5db-8345-415c-b76e-9b2a59339775\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.864820 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.864773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4808c5db-8345-415c-b76e-9b2a59339775-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-sjxjz\" (UID: \"4808c5db-8345-415c-b76e-9b2a59339775\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.864820 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.864817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq8x7\" (UniqueName: \"kubernetes.io/projected/4808c5db-8345-415c-b76e-9b2a59339775-kube-api-access-tq8x7\") pod \"payload-pre-processing-74584bb9f8-sjxjz\" (UID: \"4808c5db-8345-415c-b76e-9b2a59339775\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.865503 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.865480 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4808c5db-8345-415c-b76e-9b2a59339775-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-sjxjz\" (UID: \"4808c5db-8345-415c-b76e-9b2a59339775\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.874011 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.873979 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq8x7\" (UniqueName: \"kubernetes.io/projected/4808c5db-8345-415c-b76e-9b2a59339775-kube-api-access-tq8x7\") pod \"payload-pre-processing-74584bb9f8-sjxjz\" (UID: \"4808c5db-8345-415c-b76e-9b2a59339775\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:50.926754 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:50.926711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:51.076294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:51.076261 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz"] Jul 13 19:07:51.077842 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:07:51.077811 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4808c5db_8345_415c_b76e_9b2a59339775.slice/crio-fb770d65dd83a9b5c4ca54ada36d06c8e28d402d131cebe20cc612c55ea55f85 WatchSource:0}: Error finding container fb770d65dd83a9b5c4ca54ada36d06c8e28d402d131cebe20cc612c55ea55f85: Status 404 returned error can't find the container with id fb770d65dd83a9b5c4ca54ada36d06c8e28d402d131cebe20cc612c55ea55f85 Jul 13 19:07:51.461573 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:51.461516 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" event={"ID":"4808c5db-8345-415c-b76e-9b2a59339775","Type":"ContainerStarted","Data":"fb770d65dd83a9b5c4ca54ada36d06c8e28d402d131cebe20cc612c55ea55f85"} Jul 13 19:07:52.468540 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:52.468499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" event={"ID":"290a08de-d79d-4db1-97b3-717d202978a2","Type":"ContainerStarted","Data":"04005200573b84d8019160f18f75c73b63de9788abf9edc0ce624e5a2c873463"} Jul 13 19:07:52.468983 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:52.468630 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:07:52.487414 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:52.487361 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" podStartSLOduration=1.4782243529999999 podStartE2EDuration="3.48734328s" totalImagesPullingTime="2.009118927s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:07:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:07:52.485207005 +0000 UTC m=+576.513765345" watchObservedRunningTime="2026-07-13 19:07:52.48734328 +0000 UTC m=+576.515901617" Jul 13 19:07:54.480967 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:54.480915 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" event={"ID":"4808c5db-8345-415c-b76e-9b2a59339775","Type":"ContainerStarted","Data":"d12a7e7550471f82e8e3dde37a7394bc8fe4dd4aa8c8a2b2cae30f8031fa702c"} Jul 13 19:07:54.481368 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:54.481077 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:07:54.501172 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:54.501112 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" podStartSLOduration=2.062849793 podStartE2EDuration="4.501093792s" totalImagesPullingTime="2.438243999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:07:54.498746353 +0000 UTC m=+578.527304717" watchObservedRunningTime="2026-07-13 19:07:54.501093792 +0000 UTC m=+578.529652130" Jul 13 19:07:58.480135 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:07:58.480107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-779d67c848-9hpgg" Jul 13 19:08:05.489296 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:05.489258 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-sjxjz" Jul 13 19:08:16.446692 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:16.446636 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:08:16.446692 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:16.446661 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:08:16.452398 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:16.452375 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:08:16.452398 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:16.452384 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:08:45.255998 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.255903 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk"] Jul 13 19:08:45.258191 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.258173 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.262811 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.262786 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-j6mtd\"" Jul 13 19:08:45.262811 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.262804 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 13 19:08:45.262990 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.262838 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 13 19:08:45.262990 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.262865 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 13 19:08:45.269589 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.269563 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk"] Jul 13 19:08:45.317065 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.317024 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.317254 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.317073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.317254 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.317156 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbffl\" (UniqueName: \"kubernetes.io/projected/f19628e6-608f-40be-accd-929ccdbdc16e-kube-api-access-dbffl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.317254 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.317210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.317254 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.317235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19628e6-608f-40be-accd-929ccdbdc16e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.317413 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.317302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.417902 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.417864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.417927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.417961 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.418003 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbffl\" (UniqueName: \"kubernetes.io/projected/f19628e6-608f-40be-accd-929ccdbdc16e-kube-api-access-dbffl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.418049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.418079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19628e6-608f-40be-accd-929ccdbdc16e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418446 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.418420 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418574 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.418432 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.418640 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.418619 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.420428 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.420401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f19628e6-608f-40be-accd-929ccdbdc16e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.421136 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.421119 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f19628e6-608f-40be-accd-929ccdbdc16e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.428784 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.428755 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbffl\" (UniqueName: \"kubernetes.io/projected/f19628e6-608f-40be-accd-929ccdbdc16e-kube-api-access-dbffl\") pod \"e2e-trlp-test-simulated-kserve-598466548d-8nmmk\" (UID: \"f19628e6-608f-40be-accd-929ccdbdc16e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.568712 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.568593 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:45.707527 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.707496 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk"] Jul 13 19:08:45.711182 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:08:45.710960 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf19628e6_608f_40be_accd_929ccdbdc16e.slice/crio-fb4702ace63c28193d1945689100aceb7ae787fccbbfed5b80a2aeee7ee71246 WatchSource:0}: Error finding container fb4702ace63c28193d1945689100aceb7ae787fccbbfed5b80a2aeee7ee71246: Status 404 returned error can't find the container with id fb4702ace63c28193d1945689100aceb7ae787fccbbfed5b80a2aeee7ee71246 Jul 13 19:08:45.712255 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.712233 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:08:45.756112 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:45.756075 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" event={"ID":"f19628e6-608f-40be-accd-929ccdbdc16e","Type":"ContainerStarted","Data":"fb4702ace63c28193d1945689100aceb7ae787fccbbfed5b80a2aeee7ee71246"} Jul 13 19:08:46.763518 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:46.763477 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" event={"ID":"f19628e6-608f-40be-accd-929ccdbdc16e","Type":"ContainerStarted","Data":"70962380af6860a49bf9297be360f9993038a7d17fc90e5cead8d34479ce3fd5"} Jul 13 19:08:46.784549 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:46.784480 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" podStartSLOduration=0.815338139 podStartE2EDuration="1.784464835s" totalImagesPullingTime="969.126696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:08:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:08:46.782463572 +0000 UTC m=+630.811021910" watchObservedRunningTime="2026-07-13 19:08:46.784464835 +0000 UTC m=+630.813023172" Jul 13 19:08:55.569733 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:55.569659 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:55.570122 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:55.569767 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:55.582478 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:55.582449 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:55.822227 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:55.822132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-8nmmk" Jul 13 19:08:56.552338 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.552301 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2"] Jul 13 19:08:56.556601 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.556573 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.559301 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.559274 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 13 19:08:56.566427 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.566400 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2"] Jul 13 19:08:56.714604 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.714558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.715048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.714620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.715048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.714703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj2p6\" (UniqueName: \"kubernetes.io/projected/3dd65527-fd50-4416-91f3-e6056cb733c6-kube-api-access-zj2p6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.715048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.714778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.715048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.714823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.715048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.714849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd65527-fd50-4416-91f3-e6056cb733c6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd65527-fd50-4416-91f3-e6056cb733c6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815755 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815478 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815755 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815518 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815755 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815552 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj2p6\" (UniqueName: \"kubernetes.io/projected/3dd65527-fd50-4416-91f3-e6056cb733c6-kube-api-access-zj2p6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815927 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815763 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.815927 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.815799 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.816154 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.816128 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.818157 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.818137 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3dd65527-fd50-4416-91f3-e6056cb733c6-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.818259 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.818228 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3dd65527-fd50-4416-91f3-e6056cb733c6-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.824892 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.824863 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj2p6\" (UniqueName: \"kubernetes.io/projected/3dd65527-fd50-4416-91f3-e6056cb733c6-kube-api-access-zj2p6\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2\" (UID: \"3dd65527-fd50-4416-91f3-e6056cb733c6\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:56.868786 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:56.868725 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:08:57.009470 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:57.009434 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2"] Jul 13 19:08:57.010810 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:08:57.010783 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd65527_fd50_4416_91f3_e6056cb733c6.slice/crio-94b6d8f8293a617316835ca4df886edc6cb11a481aa1214a5d660ce3359aec17 WatchSource:0}: Error finding container 94b6d8f8293a617316835ca4df886edc6cb11a481aa1214a5d660ce3359aec17: Status 404 returned error can't find the container with id 94b6d8f8293a617316835ca4df886edc6cb11a481aa1214a5d660ce3359aec17 Jul 13 19:08:57.823108 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:57.823065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" event={"ID":"3dd65527-fd50-4416-91f3-e6056cb733c6","Type":"ContainerStarted","Data":"866264deb37058f5a161ad187bed6c5ab28094d86ea2fdcf500cd8387faa6f66"} Jul 13 19:08:57.823479 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:57.823170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" event={"ID":"3dd65527-fd50-4416-91f3-e6056cb733c6","Type":"ContainerStarted","Data":"94b6d8f8293a617316835ca4df886edc6cb11a481aa1214a5d660ce3359aec17"} Jul 13 19:08:57.843860 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:08:57.843785 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" podStartSLOduration=1.6345073970000001 podStartE2EDuration="1.843760609s" totalImagesPullingTime="209.253212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:08:57.84129761 +0000 UTC m=+641.869855972" watchObservedRunningTime="2026-07-13 19:08:57.843760609 +0000 UTC m=+641.872318947" Jul 13 19:09:06.869816 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:06.869772 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:09:06.870320 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:06.869865 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:09:06.882890 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:06.882850 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:09:07.888543 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:07.888506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2" Jul 13 19:09:19.360280 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.360237 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4"] Jul 13 19:09:19.364627 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.364599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.367315 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.367287 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 13 19:09:19.377237 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.377208 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4"] Jul 13 19:09:19.413130 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.413075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.413324 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.413139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqnx9\" (UniqueName: \"kubernetes.io/projected/3635e5de-7a13-4bf5-b750-81e9850ddf85-kube-api-access-fqnx9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.413324 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.413183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.413324 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.413259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.413324 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.413308 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.413490 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.413382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3635e5de-7a13-4bf5-b750-81e9850ddf85-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514054 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514014 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqnx9\" (UniqueName: \"kubernetes.io/projected/3635e5de-7a13-4bf5-b750-81e9850ddf85-kube-api-access-fqnx9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514153 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3635e5de-7a13-4bf5-b750-81e9850ddf85-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514590 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514661 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514598 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.514741 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.514651 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.516729 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.516696 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3635e5de-7a13-4bf5-b750-81e9850ddf85-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.516832 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.516800 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3635e5de-7a13-4bf5-b750-81e9850ddf85-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.523813 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.523785 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqnx9\" (UniqueName: \"kubernetes.io/projected/3635e5de-7a13-4bf5-b750-81e9850ddf85-kube-api-access-fqnx9\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-cb9r4\" (UID: \"3635e5de-7a13-4bf5-b750-81e9850ddf85\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.675741 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.675640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:19.821723 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.821693 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4"] Jul 13 19:09:19.823220 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:09:19.823186 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3635e5de_7a13_4bf5_b750_81e9850ddf85.slice/crio-d52d68c83b1fef4f5e2b9dce805c143d4de97de3ee069d990d73d1415e216bb6 WatchSource:0}: Error finding container d52d68c83b1fef4f5e2b9dce805c143d4de97de3ee069d990d73d1415e216bb6: Status 404 returned error can't find the container with id d52d68c83b1fef4f5e2b9dce805c143d4de97de3ee069d990d73d1415e216bb6 Jul 13 19:09:19.941113 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:19.941024 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" event={"ID":"3635e5de-7a13-4bf5-b750-81e9850ddf85","Type":"ContainerStarted","Data":"d52d68c83b1fef4f5e2b9dce805c143d4de97de3ee069d990d73d1415e216bb6"} Jul 13 19:09:20.947766 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:20.947728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" event={"ID":"3635e5de-7a13-4bf5-b750-81e9850ddf85","Type":"ContainerStarted","Data":"4070386d752d104b15d6b175e9d7f402816c4b557c6823f17f038a9c798d97a6"} Jul 13 19:09:20.968010 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:20.967952 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" podStartSLOduration=1.704680875 podStartE2EDuration="1.967933727s" totalImagesPullingTime="263.252852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:09:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:09:20.965255397 +0000 UTC m=+664.993813734" watchObservedRunningTime="2026-07-13 19:09:20.967933727 +0000 UTC m=+664.996492064" Jul 13 19:09:29.676369 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:29.676329 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:29.676772 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:29.676396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:29.688910 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:29.688879 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:09:30.007795 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:09:30.007694 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-cb9r4" Jul 13 19:11:34.755507 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:34.755469 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54f69f5bc-pvb4r"] Jul 13 19:11:34.756081 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:34.755738 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" podUID="2c17f8bb-cf44-4b07-b267-cb95fb87f277" containerName="manager" containerID="cri-o://8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355" gracePeriod=30 Jul 13 19:11:35.012422 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.012353 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:11:35.148227 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.148184 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c17f8bb-cf44-4b07-b267-cb95fb87f277-webhook-cert\") pod \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " Jul 13 19:11:35.148227 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.148236 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lfs47\" (UniqueName: \"kubernetes.io/projected/2c17f8bb-cf44-4b07-b267-cb95fb87f277-kube-api-access-lfs47\") pod \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\" (UID: \"2c17f8bb-cf44-4b07-b267-cb95fb87f277\") " Jul 13 19:11:35.150537 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.150502 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c17f8bb-cf44-4b07-b267-cb95fb87f277-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "2c17f8bb-cf44-4b07-b267-cb95fb87f277" (UID: "2c17f8bb-cf44-4b07-b267-cb95fb87f277"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:11:35.150537 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.150520 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c17f8bb-cf44-4b07-b267-cb95fb87f277-kube-api-access-lfs47" (OuterVolumeSpecName: "kube-api-access-lfs47") pod "2c17f8bb-cf44-4b07-b267-cb95fb87f277" (UID: "2c17f8bb-cf44-4b07-b267-cb95fb87f277"). InnerVolumeSpecName "kube-api-access-lfs47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:11:35.249703 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.249640 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfs47\" (UniqueName: \"kubernetes.io/projected/2c17f8bb-cf44-4b07-b267-cb95fb87f277-kube-api-access-lfs47\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:11:35.249703 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.249710 2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c17f8bb-cf44-4b07-b267-cb95fb87f277-webhook-cert\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:11:35.657176 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.657140 2580 generic.go:356] "Generic (PLEG): container finished" podID="2c17f8bb-cf44-4b07-b267-cb95fb87f277" containerID="8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355" exitCode=0 Jul 13 19:11:35.657383 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.657186 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" Jul 13 19:11:35.657383 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.657224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" event={"ID":"2c17f8bb-cf44-4b07-b267-cb95fb87f277","Type":"ContainerDied","Data":"8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355"} Jul 13 19:11:35.657383 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.657279 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-54f69f5bc-pvb4r" event={"ID":"2c17f8bb-cf44-4b07-b267-cb95fb87f277","Type":"ContainerDied","Data":"a74bc62ca3283db61d26bd81d6442ec6e58c5446c2938ff4fd75afa29da93b13"} Jul 13 19:11:35.657383 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.657307 2580 scope.go:117] "RemoveContainer" containerID="8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355" Jul 13 19:11:35.666966 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.666945 2580 scope.go:117] "RemoveContainer" containerID="8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355" Jul 13 19:11:35.667302 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:11:35.667283 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355\": container with ID starting with 8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355 not found: ID does not exist" containerID="8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355" Jul 13 19:11:35.667351 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.667311 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355"} err="failed to get container status \"8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355\": rpc error: code = NotFound desc = could not find container \"8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355\": container with ID starting with 8786c4b5952cdfb59bf6c636ac7bf3d42674278e7902603cd08a15149f538355 not found: ID does not exist" Jul 13 19:11:35.681606 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.681569 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-54f69f5bc-pvb4r"] Jul 13 19:11:35.689454 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:35.689424 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-54f69f5bc-pvb4r"] Jul 13 19:11:36.546666 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:11:36.546632 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c17f8bb-cf44-4b07-b267-cb95fb87f277" path="/var/lib/kubelet/pods/2c17f8bb-cf44-4b07-b267-cb95fb87f277/volumes" Jul 13 19:13:16.484431 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:13:16.484346 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:13:16.485924 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:13:16.485892 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:13:16.490762 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:13:16.490734 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:13:16.492489 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:13:16.492363 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:18:16.519618 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:18:16.519590 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:18:16.522793 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:18:16.522774 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:18:16.524749 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:18:16.524731 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:18:16.527646 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:18:16.527631 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:23:16.556031 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:23:16.556001 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:23:16.561216 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:23:16.561193 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:23:16.561381 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:23:16.561370 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:23:16.566498 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:23:16.566474 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:24:01.948363 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.948329 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq"] Jul 13 19:24:01.948912 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.948754 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c17f8bb-cf44-4b07-b267-cb95fb87f277" containerName="manager" Jul 13 19:24:01.948912 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.948766 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c17f8bb-cf44-4b07-b267-cb95fb87f277" containerName="manager" Jul 13 19:24:01.948912 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.948820 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c17f8bb-cf44-4b07-b267-cb95fb87f277" containerName="manager" Jul 13 19:24:01.951969 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.951934 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:01.954550 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.954523 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0b1b2151-serving-cert\"" Jul 13 19:24:01.971241 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.971209 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq"] Jul 13 19:24:01.997242 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.997202 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-maas-api-tls\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:01.997242 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.997249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-openshift-service-ca\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:01.997452 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:01.997374 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms48p\" (UniqueName: \"kubernetes.io/projected/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-kube-api-access-ms48p\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.098850 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.098813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-openshift-service-ca\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.099038 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.098881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms48p\" (UniqueName: \"kubernetes.io/projected/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-kube-api-access-ms48p\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.099038 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.098962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-maas-api-tls\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.099543 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.099518 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-openshift-service-ca\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.101438 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.101415 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-maas-api-tls\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.112206 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.112166 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms48p\" (UniqueName: \"kubernetes.io/projected/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-kube-api-access-ms48p\") pod \"maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.263154 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.263058 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:02.433127 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.433089 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq"] Jul 13 19:24:02.438903 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:24:02.438874 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6342de7_e3d4_4cc6_b4ee_4e1cdefeecc7.slice/crio-ff270372454fd83b154ec9cdf70e8af23bc488ef7e3c670ebbe2eaf51d18da0b WatchSource:0}: Error finding container ff270372454fd83b154ec9cdf70e8af23bc488ef7e3c670ebbe2eaf51d18da0b: Status 404 returned error can't find the container with id ff270372454fd83b154ec9cdf70e8af23bc488ef7e3c670ebbe2eaf51d18da0b Jul 13 19:24:02.440352 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.440332 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:24:02.587382 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:02.587344 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" event={"ID":"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7","Type":"ContainerStarted","Data":"ff270372454fd83b154ec9cdf70e8af23bc488ef7e3c670ebbe2eaf51d18da0b"} Jul 13 19:24:03.594063 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:03.594026 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" event={"ID":"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7","Type":"ContainerStarted","Data":"9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8"} Jul 13 19:24:03.594505 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:03.594145 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:03.616234 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:03.616155 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" podStartSLOduration=2.065221804 podStartE2EDuration="2.616138481s" totalImagesPullingTime="550.916677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:24:03.613345708 +0000 UTC m=+1547.641904080" watchObservedRunningTime="2026-07-13 19:24:03.616138481 +0000 UTC m=+1547.644696817" Jul 13 19:24:09.605728 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:09.605683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:17.583444 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.583402 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq"] Jul 13 19:24:17.583903 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.583651 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" podUID="c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" containerName="maas-api" containerID="cri-o://9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8" gracePeriod=30 Jul 13 19:24:17.842631 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.842566 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:17.946854 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.946812 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-openshift-service-ca\") pod \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " Jul 13 19:24:17.947058 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.946862 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ms48p\" (UniqueName: \"kubernetes.io/projected/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-kube-api-access-ms48p\") pod \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " Jul 13 19:24:17.947058 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.946909 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-maas-api-tls\") pod \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\" (UID: \"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7\") " Jul 13 19:24:17.947217 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.947194 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" (UID: "c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:24:17.949188 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.949156 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-kube-api-access-ms48p" (OuterVolumeSpecName: "kube-api-access-ms48p") pod "c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" (UID: "c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7"). InnerVolumeSpecName "kube-api-access-ms48p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:24:17.949300 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:17.949259 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" (UID: "c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:24:18.047901 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.047854 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ms48p\" (UniqueName: \"kubernetes.io/projected/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-kube-api-access-ms48p\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:24:18.047901 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.047890 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:24:18.047901 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.047900 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:24:18.678128 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.678090 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" containerID="9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8" exitCode=0 Jul 13 19:24:18.678617 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.678143 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" Jul 13 19:24:18.678617 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.678177 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" event={"ID":"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7","Type":"ContainerDied","Data":"9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8"} Jul 13 19:24:18.678617 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.678229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq" event={"ID":"c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7","Type":"ContainerDied","Data":"ff270372454fd83b154ec9cdf70e8af23bc488ef7e3c670ebbe2eaf51d18da0b"} Jul 13 19:24:18.678617 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.678252 2580 scope.go:117] "RemoveContainer" containerID="9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8" Jul 13 19:24:18.687667 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.687648 2580 scope.go:117] "RemoveContainer" containerID="9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8" Jul 13 19:24:18.688077 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:24:18.688057 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8\": container with ID starting with 9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8 not found: ID does not exist" containerID="9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8" Jul 13 19:24:18.688351 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.688088 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8"} err="failed to get container status \"9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8\": rpc error: code = NotFound desc = could not find container \"9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8\": container with ID starting with 9491148102fe37bfb7a0df8daf9ab13b6dcceceeb27ab70a2b37db94dd1509d8 not found: ID does not exist" Jul 13 19:24:18.696966 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.696929 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq"] Jul 13 19:24:18.702902 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:18.702871 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b1b2151-59d58467fc-mjkkq"] Jul 13 19:24:20.554357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:20.554312 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" path="/var/lib/kubelet/pods/c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7/volumes" Jul 13 19:24:34.313046 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.313007 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk"] Jul 13 19:24:34.313419 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.313398 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" containerName="maas-api" Jul 13 19:24:34.313419 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.313410 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" containerName="maas-api" Jul 13 19:24:34.313492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.313480 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6342de7-e3d4-4cc6-b4ee-4e1cdefeecc7" containerName="maas-api" Jul 13 19:24:34.317851 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.317822 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.321493 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.321468 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-481cf1cb-serving-cert\"" Jul 13 19:24:34.343966 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.343922 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk"] Jul 13 19:24:34.397992 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.397948 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48f2w\" (UniqueName: \"kubernetes.io/projected/22cf2ee2-9312-4ddf-a92a-83668369eda8-kube-api-access-48f2w\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.398158 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.398108 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.398158 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.398150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22cf2ee2-9312-4ddf-a92a-83668369eda8-openshift-service-ca\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.499423 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.499378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.499631 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.499435 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22cf2ee2-9312-4ddf-a92a-83668369eda8-openshift-service-ca\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.499631 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.499496 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48f2w\" (UniqueName: \"kubernetes.io/projected/22cf2ee2-9312-4ddf-a92a-83668369eda8-kube-api-access-48f2w\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.499631 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:24:34.499520 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-serving-cert: secret "maas-api-e2e-migrate-481cf1cb-serving-cert" not found Jul 13 19:24:34.499631 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:24:34.499582 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls podName:22cf2ee2-9312-4ddf-a92a-83668369eda8 nodeName:}" failed. No retries permitted until 2026-07-13 19:24:34.999566482 +0000 UTC m=+1579.028124797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls") pod "maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" (UID: "22cf2ee2-9312-4ddf-a92a-83668369eda8") : secret "maas-api-e2e-migrate-481cf1cb-serving-cert" not found Jul 13 19:24:34.500067 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.500047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22cf2ee2-9312-4ddf-a92a-83668369eda8-openshift-service-ca\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:34.517307 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:34.517268 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48f2w\" (UniqueName: \"kubernetes.io/projected/22cf2ee2-9312-4ddf-a92a-83668369eda8-kube-api-access-48f2w\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:35.003968 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:35.003926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:35.006413 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:35.006392 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls\") pod \"maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:35.228746 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:35.228712 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:35.388303 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:35.388276 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk"] Jul 13 19:24:35.389797 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:24:35.389773 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22cf2ee2_9312_4ddf_a92a_83668369eda8.slice/crio-843dbf4e9fc27d5a3bf30f5b549ba8ab5f63bca43787e4565000d6a497a9cfa8 WatchSource:0}: Error finding container 843dbf4e9fc27d5a3bf30f5b549ba8ab5f63bca43787e4565000d6a497a9cfa8: Status 404 returned error can't find the container with id 843dbf4e9fc27d5a3bf30f5b549ba8ab5f63bca43787e4565000d6a497a9cfa8 Jul 13 19:24:35.774709 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:35.774651 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" event={"ID":"22cf2ee2-9312-4ddf-a92a-83668369eda8","Type":"ContainerStarted","Data":"843dbf4e9fc27d5a3bf30f5b549ba8ab5f63bca43787e4565000d6a497a9cfa8"} Jul 13 19:24:36.781091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:36.781061 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" event={"ID":"22cf2ee2-9312-4ddf-a92a-83668369eda8","Type":"ContainerStarted","Data":"3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313"} Jul 13 19:24:36.781509 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:36.781167 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:42.792452 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:42.792418 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:24:42.814773 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:24:42.814721 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" podStartSLOduration=8.382465494 podStartE2EDuration="8.814704365s" totalImagesPullingTime="432.238871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:24:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:24:36.826688936 +0000 UTC m=+1580.855247278" watchObservedRunningTime="2026-07-13 19:24:42.814704365 +0000 UTC m=+1586.843262702" Jul 13 19:25:39.118597 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.118562 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk"] Jul 13 19:25:39.121202 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.119282 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" podUID="22cf2ee2-9312-4ddf-a92a-83668369eda8" containerName="maas-api" containerID="cri-o://3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313" gracePeriod=30 Jul 13 19:25:39.372196 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.372123 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:25:39.473086 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.473050 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48f2w\" (UniqueName: \"kubernetes.io/projected/22cf2ee2-9312-4ddf-a92a-83668369eda8-kube-api-access-48f2w\") pod \"22cf2ee2-9312-4ddf-a92a-83668369eda8\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " Jul 13 19:25:39.473308 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.473097 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22cf2ee2-9312-4ddf-a92a-83668369eda8-openshift-service-ca\") pod \"22cf2ee2-9312-4ddf-a92a-83668369eda8\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " Jul 13 19:25:39.473308 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.473161 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls\") pod \"22cf2ee2-9312-4ddf-a92a-83668369eda8\" (UID: \"22cf2ee2-9312-4ddf-a92a-83668369eda8\") " Jul 13 19:25:39.473510 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.473486 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22cf2ee2-9312-4ddf-a92a-83668369eda8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "22cf2ee2-9312-4ddf-a92a-83668369eda8" (UID: "22cf2ee2-9312-4ddf-a92a-83668369eda8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:25:39.475302 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.475271 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "22cf2ee2-9312-4ddf-a92a-83668369eda8" (UID: "22cf2ee2-9312-4ddf-a92a-83668369eda8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:25:39.475411 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.475318 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22cf2ee2-9312-4ddf-a92a-83668369eda8-kube-api-access-48f2w" (OuterVolumeSpecName: "kube-api-access-48f2w") pod "22cf2ee2-9312-4ddf-a92a-83668369eda8" (UID: "22cf2ee2-9312-4ddf-a92a-83668369eda8"). InnerVolumeSpecName "kube-api-access-48f2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:25:39.574168 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.574112 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48f2w\" (UniqueName: \"kubernetes.io/projected/22cf2ee2-9312-4ddf-a92a-83668369eda8-kube-api-access-48f2w\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:25:39.574168 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.574159 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/22cf2ee2-9312-4ddf-a92a-83668369eda8-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:25:39.574168 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:39.574170 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/22cf2ee2-9312-4ddf-a92a-83668369eda8-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:25:40.130810 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.130778 2580 generic.go:356] "Generic (PLEG): container finished" podID="22cf2ee2-9312-4ddf-a92a-83668369eda8" containerID="3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313" exitCode=0 Jul 13 19:25:40.131236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.130832 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" Jul 13 19:25:40.131236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.130862 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" event={"ID":"22cf2ee2-9312-4ddf-a92a-83668369eda8","Type":"ContainerDied","Data":"3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313"} Jul 13 19:25:40.131236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.130914 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk" event={"ID":"22cf2ee2-9312-4ddf-a92a-83668369eda8","Type":"ContainerDied","Data":"843dbf4e9fc27d5a3bf30f5b549ba8ab5f63bca43787e4565000d6a497a9cfa8"} Jul 13 19:25:40.131236 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.130940 2580 scope.go:117] "RemoveContainer" containerID="3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313" Jul 13 19:25:40.140802 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.140783 2580 scope.go:117] "RemoveContainer" containerID="3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313" Jul 13 19:25:40.141104 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:25:40.141082 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313\": container with ID starting with 3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313 not found: ID does not exist" containerID="3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313" Jul 13 19:25:40.141162 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.141114 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313"} err="failed to get container status \"3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313\": rpc error: code = NotFound desc = could not find container \"3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313\": container with ID starting with 3ae1efb44d714c7d89b0e699dcb3a8c4f7b716db1fad1f3761fdd6656ed5c313 not found: ID does not exist" Jul 13 19:25:40.154035 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.153990 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk"] Jul 13 19:25:40.157891 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.157862 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-481cf1cb-5f5bf8d4c4-g4hqk"] Jul 13 19:25:40.546434 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:40.546351 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22cf2ee2-9312-4ddf-a92a-83668369eda8" path="/var/lib/kubelet/pods/22cf2ee2-9312-4ddf-a92a-83668369eda8/volumes" Jul 13 19:25:46.680856 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.680818 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v"] Jul 13 19:25:46.681343 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.681322 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22cf2ee2-9312-4ddf-a92a-83668369eda8" containerName="maas-api" Jul 13 19:25:46.681405 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.681346 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="22cf2ee2-9312-4ddf-a92a-83668369eda8" containerName="maas-api" Jul 13 19:25:46.681458 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.681433 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="22cf2ee2-9312-4ddf-a92a-83668369eda8" containerName="maas-api" Jul 13 19:25:46.686182 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.686158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.690102 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.690079 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-84c88922-serving-cert\"" Jul 13 19:25:46.699018 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.698994 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v"] Jul 13 19:25:46.735890 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.735848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.736126 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.735958 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhmjh\" (UniqueName: \"kubernetes.io/projected/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-kube-api-access-nhmjh\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.736126 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.735983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-openshift-service-ca\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.837367 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.837315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.837561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.837412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhmjh\" (UniqueName: \"kubernetes.io/projected/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-kube-api-access-nhmjh\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.837561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.837440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-openshift-service-ca\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.837561 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:25:46.837458 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-serving-cert: secret "maas-api-e2e-ait-84c88922-serving-cert" not found Jul 13 19:25:46.837561 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:25:46.837518 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls podName:fe608de0-eb81-4653-94f1-3b3d31ae3f4e nodeName:}" failed. No retries permitted until 2026-07-13 19:25:47.337501129 +0000 UTC m=+1651.366059444 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls") pod "maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" (UID: "fe608de0-eb81-4653-94f1-3b3d31ae3f4e") : secret "maas-api-e2e-ait-84c88922-serving-cert" not found Jul 13 19:25:46.838221 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.838201 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-openshift-service-ca\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:46.846782 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:46.846757 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhmjh\" (UniqueName: \"kubernetes.io/projected/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-kube-api-access-nhmjh\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:47.340986 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:47.340951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:47.343496 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:47.343471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls\") pod \"maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:47.598995 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:47.598890 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:47.747545 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:47.747517 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v"] Jul 13 19:25:47.749303 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:25:47.749267 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe608de0_eb81_4653_94f1_3b3d31ae3f4e.slice/crio-bb5694aeefa2749deebd2aeb49ec83849f6ded426b1b6f7713844b46aa341eaa WatchSource:0}: Error finding container bb5694aeefa2749deebd2aeb49ec83849f6ded426b1b6f7713844b46aa341eaa: Status 404 returned error can't find the container with id bb5694aeefa2749deebd2aeb49ec83849f6ded426b1b6f7713844b46aa341eaa Jul 13 19:25:48.175145 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:48.175113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" event={"ID":"fe608de0-eb81-4653-94f1-3b3d31ae3f4e","Type":"ContainerStarted","Data":"bb5694aeefa2749deebd2aeb49ec83849f6ded426b1b6f7713844b46aa341eaa"} Jul 13 19:25:49.181743 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:49.181623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" event={"ID":"fe608de0-eb81-4653-94f1-3b3d31ae3f4e","Type":"ContainerStarted","Data":"437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63"} Jul 13 19:25:49.181743 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:49.181709 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:25:49.203079 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:49.203019 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" podStartSLOduration=2.770736129 podStartE2EDuration="3.202997664s" totalImagesPullingTime="432.261535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:25:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:25:49.201011622 +0000 UTC m=+1653.229569985" watchObservedRunningTime="2026-07-13 19:25:49.202997664 +0000 UTC m=+1653.231556001" Jul 13 19:25:55.192241 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:25:55.192211 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:26:52.236688 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.236578 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v"] Jul 13 19:26:52.237813 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.236876 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" podUID="fe608de0-eb81-4653-94f1-3b3d31ae3f4e" containerName="maas-api" containerID="cri-o://437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63" gracePeriod=30 Jul 13 19:26:52.489951 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.489874 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:26:52.522332 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.522287 2580 generic.go:356] "Generic (PLEG): container finished" podID="fe608de0-eb81-4653-94f1-3b3d31ae3f4e" containerID="437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63" exitCode=0 Jul 13 19:26:52.522501 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.522352 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" Jul 13 19:26:52.522501 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.522376 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" event={"ID":"fe608de0-eb81-4653-94f1-3b3d31ae3f4e","Type":"ContainerDied","Data":"437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63"} Jul 13 19:26:52.522501 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.522442 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v" event={"ID":"fe608de0-eb81-4653-94f1-3b3d31ae3f4e","Type":"ContainerDied","Data":"bb5694aeefa2749deebd2aeb49ec83849f6ded426b1b6f7713844b46aa341eaa"} Jul 13 19:26:52.522501 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.522472 2580 scope.go:117] "RemoveContainer" containerID="437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63" Jul 13 19:26:52.531714 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.531683 2580 scope.go:117] "RemoveContainer" containerID="437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63" Jul 13 19:26:52.532069 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:26:52.532049 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63\": container with ID starting with 437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63 not found: ID does not exist" containerID="437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63" Jul 13 19:26:52.532155 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.532076 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63"} err="failed to get container status \"437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63\": rpc error: code = NotFound desc = could not find container \"437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63\": container with ID starting with 437d88d5227bfbb86f252a19905ac103c0f997cf2fa3cb33fe8d912005f97e63 not found: ID does not exist" Jul 13 19:26:52.631146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.631103 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhmjh\" (UniqueName: \"kubernetes.io/projected/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-kube-api-access-nhmjh\") pod \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " Jul 13 19:26:52.631353 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.631165 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-openshift-service-ca\") pod \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " Jul 13 19:26:52.631353 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.631195 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls\") pod \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\" (UID: \"fe608de0-eb81-4653-94f1-3b3d31ae3f4e\") " Jul 13 19:26:52.631599 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.631571 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe608de0-eb81-4653-94f1-3b3d31ae3f4e" (UID: "fe608de0-eb81-4653-94f1-3b3d31ae3f4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:26:52.631949 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.631915 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:26:52.633476 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.633453 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe608de0-eb81-4653-94f1-3b3d31ae3f4e" (UID: "fe608de0-eb81-4653-94f1-3b3d31ae3f4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:26:52.633569 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.633498 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-kube-api-access-nhmjh" (OuterVolumeSpecName: "kube-api-access-nhmjh") pod "fe608de0-eb81-4653-94f1-3b3d31ae3f4e" (UID: "fe608de0-eb81-4653-94f1-3b3d31ae3f4e"). InnerVolumeSpecName "kube-api-access-nhmjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:26:52.732684 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.732621 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhmjh\" (UniqueName: \"kubernetes.io/projected/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-kube-api-access-nhmjh\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:26:52.732684 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.732663 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe608de0-eb81-4653-94f1-3b3d31ae3f4e-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:26:52.847731 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.847693 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v"] Jul 13 19:26:52.855163 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:52.855135 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-84c88922-6fc5b8bd4c-g647v"] Jul 13 19:26:54.546483 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:26:54.546448 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe608de0-eb81-4653-94f1-3b3d31ae3f4e" path="/var/lib/kubelet/pods/fe608de0-eb81-4653-94f1-3b3d31ae3f4e/volumes" Jul 13 19:27:09.466868 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.466834 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r"] Jul 13 19:27:09.467364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.467196 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe608de0-eb81-4653-94f1-3b3d31ae3f4e" containerName="maas-api" Jul 13 19:27:09.467364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.467207 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe608de0-eb81-4653-94f1-3b3d31ae3f4e" containerName="maas-api" Jul 13 19:27:09.467364 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.467277 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe608de0-eb81-4653-94f1-3b3d31ae3f4e" containerName="maas-api" Jul 13 19:27:09.472118 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.472087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.474907 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.474882 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-c7ee59ed-serving-cert\"" Jul 13 19:27:09.485832 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.485798 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r"] Jul 13 19:27:09.587372 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.587334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzbfq\" (UniqueName: \"kubernetes.io/projected/576413dc-961e-4ddf-b103-69f04331c76a-kube-api-access-bzbfq\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.587548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.587393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.587548 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.587484 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576413dc-961e-4ddf-b103-69f04331c76a-openshift-service-ca\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.688206 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.688170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzbfq\" (UniqueName: \"kubernetes.io/projected/576413dc-961e-4ddf-b103-69f04331c76a-kube-api-access-bzbfq\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.688414 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.688235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.688414 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.688262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576413dc-961e-4ddf-b103-69f04331c76a-openshift-service-ca\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.688414 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:27:09.688362 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-serving-cert: secret "maas-api-e2e-derive-c7ee59ed-serving-cert" not found Jul 13 19:27:09.688580 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:27:09.688421 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls podName:576413dc-961e-4ddf-b103-69f04331c76a nodeName:}" failed. No retries permitted until 2026-07-13 19:27:10.188406446 +0000 UTC m=+1734.216964761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls") pod "maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" (UID: "576413dc-961e-4ddf-b103-69f04331c76a") : secret "maas-api-e2e-derive-c7ee59ed-serving-cert" not found Jul 13 19:27:09.688866 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.688849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576413dc-961e-4ddf-b103-69f04331c76a-openshift-service-ca\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:09.713727 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:09.713692 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzbfq\" (UniqueName: \"kubernetes.io/projected/576413dc-961e-4ddf-b103-69f04331c76a-kube-api-access-bzbfq\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:10.192135 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:10.192090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:10.194530 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:10.194507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls\") pod \"maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:10.383345 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:10.383303 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:10.535338 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:27:10.535301 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod576413dc_961e_4ddf_b103_69f04331c76a.slice/crio-8a55c125557a2e83e22698f61831c482eef0fc0522266b9bfc427d7e6dface23 WatchSource:0}: Error finding container 8a55c125557a2e83e22698f61831c482eef0fc0522266b9bfc427d7e6dface23: Status 404 returned error can't find the container with id 8a55c125557a2e83e22698f61831c482eef0fc0522266b9bfc427d7e6dface23 Jul 13 19:27:10.535765 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:10.535372 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r"] Jul 13 19:27:10.629166 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:10.629123 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" event={"ID":"576413dc-961e-4ddf-b103-69f04331c76a","Type":"ContainerStarted","Data":"8a55c125557a2e83e22698f61831c482eef0fc0522266b9bfc427d7e6dface23"} Jul 13 19:27:11.635745 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:11.635704 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" event={"ID":"576413dc-961e-4ddf-b103-69f04331c76a","Type":"ContainerStarted","Data":"1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f"} Jul 13 19:27:11.636243 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:11.635846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:27:11.659831 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:11.659780 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" podStartSLOduration=2.159738929 podStartE2EDuration="2.659762433s" totalImagesPullingTime="500.023504ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:27:11.657037329 +0000 UTC m=+1735.685595684" watchObservedRunningTime="2026-07-13 19:27:11.659762433 +0000 UTC m=+1735.688320769" Jul 13 19:27:17.645960 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:27:17.645929 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:28:14.129085 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.129038 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r"] Jul 13 19:28:14.129688 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.129277 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" podUID="576413dc-961e-4ddf-b103-69f04331c76a" containerName="maas-api" containerID="cri-o://1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f" gracePeriod=30 Jul 13 19:28:14.390392 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.390320 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:28:14.549304 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.549257 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls\") pod \"576413dc-961e-4ddf-b103-69f04331c76a\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " Jul 13 19:28:14.549515 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.549390 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576413dc-961e-4ddf-b103-69f04331c76a-openshift-service-ca\") pod \"576413dc-961e-4ddf-b103-69f04331c76a\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " Jul 13 19:28:14.549515 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.549451 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzbfq\" (UniqueName: \"kubernetes.io/projected/576413dc-961e-4ddf-b103-69f04331c76a-kube-api-access-bzbfq\") pod \"576413dc-961e-4ddf-b103-69f04331c76a\" (UID: \"576413dc-961e-4ddf-b103-69f04331c76a\") " Jul 13 19:28:14.549900 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.549867 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/576413dc-961e-4ddf-b103-69f04331c76a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "576413dc-961e-4ddf-b103-69f04331c76a" (UID: "576413dc-961e-4ddf-b103-69f04331c76a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:28:14.551870 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.551825 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/576413dc-961e-4ddf-b103-69f04331c76a-kube-api-access-bzbfq" (OuterVolumeSpecName: "kube-api-access-bzbfq") pod "576413dc-961e-4ddf-b103-69f04331c76a" (UID: "576413dc-961e-4ddf-b103-69f04331c76a"). InnerVolumeSpecName "kube-api-access-bzbfq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:28:14.552115 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.552084 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "576413dc-961e-4ddf-b103-69f04331c76a" (UID: "576413dc-961e-4ddf-b103-69f04331c76a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:28:14.650655 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.650563 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/576413dc-961e-4ddf-b103-69f04331c76a-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:28:14.650655 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.650596 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/576413dc-961e-4ddf-b103-69f04331c76a-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:28:14.650655 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.650607 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzbfq\" (UniqueName: \"kubernetes.io/projected/576413dc-961e-4ddf-b103-69f04331c76a-kube-api-access-bzbfq\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:28:14.976304 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.976210 2580 generic.go:356] "Generic (PLEG): container finished" podID="576413dc-961e-4ddf-b103-69f04331c76a" containerID="1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f" exitCode=0 Jul 13 19:28:14.976304 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.976252 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" event={"ID":"576413dc-961e-4ddf-b103-69f04331c76a","Type":"ContainerDied","Data":"1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f"} Jul 13 19:28:14.976304 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.976277 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" Jul 13 19:28:14.976304 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.976299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r" event={"ID":"576413dc-961e-4ddf-b103-69f04331c76a","Type":"ContainerDied","Data":"8a55c125557a2e83e22698f61831c482eef0fc0522266b9bfc427d7e6dface23"} Jul 13 19:28:14.976584 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.976320 2580 scope.go:117] "RemoveContainer" containerID="1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f" Jul 13 19:28:14.985643 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.985621 2580 scope.go:117] "RemoveContainer" containerID="1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f" Jul 13 19:28:14.986044 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:28:14.986026 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f\": container with ID starting with 1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f not found: ID does not exist" containerID="1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f" Jul 13 19:28:14.986104 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.986051 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f"} err="failed to get container status \"1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f\": rpc error: code = NotFound desc = could not find container \"1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f\": container with ID starting with 1b581e93cb95cdb667491d70f029a3866217a85f4c8f5f189f4133f6b94da29f not found: ID does not exist" Jul 13 19:28:14.995529 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.995503 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r"] Jul 13 19:28:14.997591 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:14.997568 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-c7ee59ed-b86684bdc-mkw9r"] Jul 13 19:28:16.547314 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:16.547279 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="576413dc-961e-4ddf-b103-69f04331c76a" path="/var/lib/kubelet/pods/576413dc-961e-4ddf-b103-69f04331c76a/volumes" Jul 13 19:28:16.589252 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:16.589226 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:28:16.595488 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:16.595459 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:28:16.596146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:16.596128 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:28:16.602405 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:28:16.602384 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:30:10.268552 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.268460 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r"] Jul 13 19:30:10.269098 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.268905 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="576413dc-961e-4ddf-b103-69f04331c76a" containerName="maas-api" Jul 13 19:30:10.269098 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.268921 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="576413dc-961e-4ddf-b103-69f04331c76a" containerName="maas-api" Jul 13 19:30:10.269098 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.269002 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="576413dc-961e-4ddf-b103-69f04331c76a" containerName="maas-api" Jul 13 19:30:10.272114 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.272095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.278012 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.277980 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d741939-ea5b-425e-9fa7-dcc91ef00814-openshift-service-ca\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.278146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.278080 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmz9v\" (UniqueName: \"kubernetes.io/projected/3d741939-ea5b-425e-9fa7-dcc91ef00814-kube-api-access-jmz9v\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.278217 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.278154 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.281980 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.281958 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-be9a1497-serving-cert\"" Jul 13 19:30:10.303050 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.303011 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r"] Jul 13 19:30:10.379565 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.379534 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d741939-ea5b-425e-9fa7-dcc91ef00814-openshift-service-ca\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.379789 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.379589 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmz9v\" (UniqueName: \"kubernetes.io/projected/3d741939-ea5b-425e-9fa7-dcc91ef00814-kube-api-access-jmz9v\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.379789 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.379641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.379864 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:30:10.379793 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-serving-cert: secret "maas-api-e2e-mt-be9a1497-serving-cert" not found Jul 13 19:30:10.379909 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:30:10.379883 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls podName:3d741939-ea5b-425e-9fa7-dcc91ef00814 nodeName:}" failed. No retries permitted until 2026-07-13 19:30:10.879861275 +0000 UTC m=+1914.908419607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls") pod "maas-api-e2e-mt-be9a1497-587649f496-tc98r" (UID: "3d741939-ea5b-425e-9fa7-dcc91ef00814") : secret "maas-api-e2e-mt-be9a1497-serving-cert" not found Jul 13 19:30:10.380337 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.380317 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d741939-ea5b-425e-9fa7-dcc91ef00814-openshift-service-ca\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.395741 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.395713 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmz9v\" (UniqueName: \"kubernetes.io/projected/3d741939-ea5b-425e-9fa7-dcc91ef00814-kube-api-access-jmz9v\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.884318 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.884275 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:10.886766 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:10.886746 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls\") pod \"maas-api-e2e-mt-be9a1497-587649f496-tc98r\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:11.183974 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:11.183845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:11.331494 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:11.331466 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r"] Jul 13 19:30:11.332384 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:30:11.332361 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d741939_ea5b_425e_9fa7_dcc91ef00814.slice/crio-dc44caf760e5b57c7c2d5a84fb7d7304cc8d0e22ae9387d78d2a46c975a877c9 WatchSource:0}: Error finding container dc44caf760e5b57c7c2d5a84fb7d7304cc8d0e22ae9387d78d2a46c975a877c9: Status 404 returned error can't find the container with id dc44caf760e5b57c7c2d5a84fb7d7304cc8d0e22ae9387d78d2a46c975a877c9 Jul 13 19:30:11.333630 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:11.333611 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:30:11.588902 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:11.588863 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" event={"ID":"3d741939-ea5b-425e-9fa7-dcc91ef00814","Type":"ContainerStarted","Data":"dc44caf760e5b57c7c2d5a84fb7d7304cc8d0e22ae9387d78d2a46c975a877c9"} Jul 13 19:30:12.595429 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:12.595389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" event={"ID":"3d741939-ea5b-425e-9fa7-dcc91ef00814","Type":"ContainerStarted","Data":"6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c"} Jul 13 19:30:12.595908 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:12.595466 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:12.616064 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:12.616013 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" podStartSLOduration=1.5204840339999999 podStartE2EDuration="2.615996418s" totalImagesPullingTime="1.095512384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:30:12.615488179 +0000 UTC m=+1916.644046527" watchObservedRunningTime="2026-07-13 19:30:12.615996418 +0000 UTC m=+1916.644554753" Jul 13 19:30:18.608632 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:18.608598 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:32.388058 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.388022 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r"] Jul 13 19:30:32.388492 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.388373 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" podUID="3d741939-ea5b-425e-9fa7-dcc91ef00814" containerName="maas-api" containerID="cri-o://6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c" gracePeriod=30 Jul 13 19:30:32.647093 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.647017 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:32.700210 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.700174 2580 generic.go:356] "Generic (PLEG): container finished" podID="3d741939-ea5b-425e-9fa7-dcc91ef00814" containerID="6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c" exitCode=0 Jul 13 19:30:32.700424 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.700227 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" Jul 13 19:30:32.700424 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.700276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" event={"ID":"3d741939-ea5b-425e-9fa7-dcc91ef00814","Type":"ContainerDied","Data":"6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c"} Jul 13 19:30:32.700424 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.700344 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r" event={"ID":"3d741939-ea5b-425e-9fa7-dcc91ef00814","Type":"ContainerDied","Data":"dc44caf760e5b57c7c2d5a84fb7d7304cc8d0e22ae9387d78d2a46c975a877c9"} Jul 13 19:30:32.700424 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.700376 2580 scope.go:117] "RemoveContainer" containerID="6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c" Jul 13 19:30:32.709262 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.709239 2580 scope.go:117] "RemoveContainer" containerID="6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c" Jul 13 19:30:32.709549 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:30:32.709530 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c\": container with ID starting with 6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c not found: ID does not exist" containerID="6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c" Jul 13 19:30:32.709633 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.709561 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c"} err="failed to get container status \"6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c\": rpc error: code = NotFound desc = could not find container \"6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c\": container with ID starting with 6ebafd7a3584c51001011d7e6a6990c5e63bd2f3a658f41e536fdf7eae2d712c not found: ID does not exist" Jul 13 19:30:32.785258 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.785222 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls\") pod \"3d741939-ea5b-425e-9fa7-dcc91ef00814\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " Jul 13 19:30:32.785497 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.785316 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmz9v\" (UniqueName: \"kubernetes.io/projected/3d741939-ea5b-425e-9fa7-dcc91ef00814-kube-api-access-jmz9v\") pod \"3d741939-ea5b-425e-9fa7-dcc91ef00814\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " Jul 13 19:30:32.785497 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.785391 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d741939-ea5b-425e-9fa7-dcc91ef00814-openshift-service-ca\") pod \"3d741939-ea5b-425e-9fa7-dcc91ef00814\" (UID: \"3d741939-ea5b-425e-9fa7-dcc91ef00814\") " Jul 13 19:30:32.785800 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.785775 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d741939-ea5b-425e-9fa7-dcc91ef00814-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d741939-ea5b-425e-9fa7-dcc91ef00814" (UID: "3d741939-ea5b-425e-9fa7-dcc91ef00814"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:30:32.787361 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.787333 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d741939-ea5b-425e-9fa7-dcc91ef00814" (UID: "3d741939-ea5b-425e-9fa7-dcc91ef00814"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:30:32.787458 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.787407 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d741939-ea5b-425e-9fa7-dcc91ef00814-kube-api-access-jmz9v" (OuterVolumeSpecName: "kube-api-access-jmz9v") pod "3d741939-ea5b-425e-9fa7-dcc91ef00814" (UID: "3d741939-ea5b-425e-9fa7-dcc91ef00814"). InnerVolumeSpecName "kube-api-access-jmz9v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:30:32.886363 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.886322 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d741939-ea5b-425e-9fa7-dcc91ef00814-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:30:32.886541 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.886381 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmz9v\" (UniqueName: \"kubernetes.io/projected/3d741939-ea5b-425e-9fa7-dcc91ef00814-kube-api-access-jmz9v\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:30:32.886541 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:32.886390 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d741939-ea5b-425e-9fa7-dcc91ef00814-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:30:33.024122 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:33.024082 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r"] Jul 13 19:30:33.029974 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:33.029940 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-be9a1497-587649f496-tc98r"] Jul 13 19:30:34.547783 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:30:34.547745 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d741939-ea5b-425e-9fa7-dcc91ef00814" path="/var/lib/kubelet/pods/3d741939-ea5b-425e-9fa7-dcc91ef00814/volumes" Jul 13 19:31:11.558784 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.558748 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn"] Jul 13 19:31:11.559289 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.559274 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d741939-ea5b-425e-9fa7-dcc91ef00814" containerName="maas-api" Jul 13 19:31:11.559337 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.559292 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d741939-ea5b-425e-9fa7-dcc91ef00814" containerName="maas-api" Jul 13 19:31:11.559388 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.559378 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d741939-ea5b-425e-9fa7-dcc91ef00814" containerName="maas-api" Jul 13 19:31:11.562613 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.562590 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.566220 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.566189 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-97bb288c-serving-cert\"" Jul 13 19:31:11.584946 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.584902 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn"] Jul 13 19:31:11.645350 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.645306 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-openshift-service-ca\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.645546 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.645367 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2qvc\" (UniqueName: \"kubernetes.io/projected/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-kube-api-access-k2qvc\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.645546 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.645395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.746618 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.746582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-openshift-service-ca\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.746842 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.746624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2qvc\" (UniqueName: \"kubernetes.io/projected/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-kube-api-access-k2qvc\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.746842 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.746647 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.746842 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:31:11.746767 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-serving-cert: secret "maas-api-e2e-mt-97bb288c-serving-cert" not found Jul 13 19:31:11.746842 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:31:11.746836 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls podName:c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5 nodeName:}" failed. No retries permitted until 2026-07-13 19:31:12.246818014 +0000 UTC m=+1976.275376329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls") pod "maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" (UID: "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5") : secret "maas-api-e2e-mt-97bb288c-serving-cert" not found Jul 13 19:31:11.747357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.747329 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-openshift-service-ca\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:11.764461 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:11.764424 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2qvc\" (UniqueName: \"kubernetes.io/projected/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-kube-api-access-k2qvc\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:12.251735 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:12.251698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:12.254307 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:12.254282 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls\") pod \"maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:12.475742 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:12.475708 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:12.624487 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:12.624455 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn"] Jul 13 19:31:12.626146 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:31:12.626116 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a2c3eb_47ea_4dbe_8802_42b7ca68f9d5.slice/crio-e000b731ea533aad8bd7d68e2e2030960b9eb88373badd244c9283e26b27719a WatchSource:0}: Error finding container e000b731ea533aad8bd7d68e2e2030960b9eb88373badd244c9283e26b27719a: Status 404 returned error can't find the container with id e000b731ea533aad8bd7d68e2e2030960b9eb88373badd244c9283e26b27719a Jul 13 19:31:12.915137 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:12.915094 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" event={"ID":"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5","Type":"ContainerStarted","Data":"e000b731ea533aad8bd7d68e2e2030960b9eb88373badd244c9283e26b27719a"} Jul 13 19:31:13.922499 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:13.922460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" event={"ID":"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5","Type":"ContainerStarted","Data":"6e7ae8fbc58869337f96882c242ec1244171d0d15d23a1b9fc218d03190ea07b"} Jul 13 19:31:13.922925 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:13.922599 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:31:13.941267 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:13.941207 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" podStartSLOduration=2.421791254 podStartE2EDuration="2.941188903s" totalImagesPullingTime="519.397649ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:31:13.939335463 +0000 UTC m=+1977.967893800" watchObservedRunningTime="2026-07-13 19:31:13.941188903 +0000 UTC m=+1977.969747565" Jul 13 19:31:19.934395 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:31:19.934365 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:32:00.048437 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.048399 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn"] Jul 13 19:32:00.049103 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.048636 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" podUID="c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" containerName="maas-api" containerID="cri-o://6e7ae8fbc58869337f96882c242ec1244171d0d15d23a1b9fc218d03190ea07b" gracePeriod=30 Jul 13 19:32:00.174906 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.174870 2580 generic.go:356] "Generic (PLEG): container finished" podID="c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" containerID="6e7ae8fbc58869337f96882c242ec1244171d0d15d23a1b9fc218d03190ea07b" exitCode=0 Jul 13 19:32:00.175070 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.174934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" event={"ID":"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5","Type":"ContainerDied","Data":"6e7ae8fbc58869337f96882c242ec1244171d0d15d23a1b9fc218d03190ea07b"} Jul 13 19:32:00.298872 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.298809 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:32:00.400088 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.400051 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls\") pod \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " Jul 13 19:32:00.400260 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.400137 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2qvc\" (UniqueName: \"kubernetes.io/projected/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-kube-api-access-k2qvc\") pod \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " Jul 13 19:32:00.400260 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.400244 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-openshift-service-ca\") pod \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\" (UID: \"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5\") " Jul 13 19:32:00.400612 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.400583 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" (UID: "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:32:00.402059 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.402035 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" (UID: "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:32:00.402220 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.402194 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-kube-api-access-k2qvc" (OuterVolumeSpecName: "kube-api-access-k2qvc") pod "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" (UID: "c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5"). InnerVolumeSpecName "kube-api-access-k2qvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:32:00.501607 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.501560 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:32:00.501607 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.501595 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:32:00.501607 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:00.501605 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2qvc\" (UniqueName: \"kubernetes.io/projected/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5-kube-api-access-k2qvc\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:32:01.181851 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:01.181817 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" Jul 13 19:32:01.182303 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:01.181851 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn" event={"ID":"c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5","Type":"ContainerDied","Data":"e000b731ea533aad8bd7d68e2e2030960b9eb88373badd244c9283e26b27719a"} Jul 13 19:32:01.182303 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:01.181917 2580 scope.go:117] "RemoveContainer" containerID="6e7ae8fbc58869337f96882c242ec1244171d0d15d23a1b9fc218d03190ea07b" Jul 13 19:32:01.199884 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:01.199854 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn"] Jul 13 19:32:01.204368 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:01.204341 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-97bb288c-6fdb99c46f-qjzqn"] Jul 13 19:32:02.546828 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:32:02.546797 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" path="/var/lib/kubelet/pods/c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5/volumes" Jul 13 19:33:16.623607 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:16.623579 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:33:16.628661 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:16.628633 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:33:16.630869 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:16.630849 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:33:16.635459 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:16.635443 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:33:55.385184 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.385149 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm"] Jul 13 19:33:55.385709 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.385506 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" containerName="maas-api" Jul 13 19:33:55.385709 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.385519 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" containerName="maas-api" Jul 13 19:33:55.385709 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.385585 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5a2c3eb-47ea-4dbe-8802-42b7ca68f9d5" containerName="maas-api" Jul 13 19:33:55.388643 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.388624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.391201 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.391180 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-d0bb2f-serving-cert\"" Jul 13 19:33:55.413146 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.413116 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm"] Jul 13 19:33:55.538006 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.537963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbvq5\" (UniqueName: \"kubernetes.io/projected/46a5e2a2-6220-462b-bedb-44dd87898a0e-kube-api-access-fbvq5\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.538006 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.538009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a5e2a2-6220-462b-bedb-44dd87898a0e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.538250 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.538050 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a5e2a2-6220-462b-bedb-44dd87898a0e-maas-api-tls\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.639276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.639167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbvq5\" (UniqueName: \"kubernetes.io/projected/46a5e2a2-6220-462b-bedb-44dd87898a0e-kube-api-access-fbvq5\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.639276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.639216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a5e2a2-6220-462b-bedb-44dd87898a0e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.639518 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.639291 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a5e2a2-6220-462b-bedb-44dd87898a0e-maas-api-tls\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.639960 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.639937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a5e2a2-6220-462b-bedb-44dd87898a0e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.641882 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.641858 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a5e2a2-6220-462b-bedb-44dd87898a0e-maas-api-tls\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.648340 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.648316 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbvq5\" (UniqueName: \"kubernetes.io/projected/46a5e2a2-6220-462b-bedb-44dd87898a0e-kube-api-access-fbvq5\") pod \"maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.701997 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.701958 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:55.844196 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:55.844165 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm"] Jul 13 19:33:55.846093 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:33:55.846064 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46a5e2a2_6220_462b_bedb_44dd87898a0e.slice/crio-95c9c00d8a39e345e9fb1b58b35a5cb816175e08017ca9b5887b9ce09e06741c WatchSource:0}: Error finding container 95c9c00d8a39e345e9fb1b58b35a5cb816175e08017ca9b5887b9ce09e06741c: Status 404 returned error can't find the container with id 95c9c00d8a39e345e9fb1b58b35a5cb816175e08017ca9b5887b9ce09e06741c Jul 13 19:33:56.785807 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:56.785771 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" event={"ID":"46a5e2a2-6220-462b-bedb-44dd87898a0e","Type":"ContainerStarted","Data":"95c9c00d8a39e345e9fb1b58b35a5cb816175e08017ca9b5887b9ce09e06741c"} Jul 13 19:33:58.798253 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:58.798208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" event={"ID":"46a5e2a2-6220-462b-bedb-44dd87898a0e","Type":"ContainerStarted","Data":"2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977"} Jul 13 19:33:58.798655 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:58.798286 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:33:58.819239 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:33:58.819191 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" podStartSLOduration=1.808430409 podStartE2EDuration="3.819175911s" totalImagesPullingTime="2.010745502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:33:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:33:58.818115771 +0000 UTC m=+2142.846674127" watchObservedRunningTime="2026-07-13 19:33:58.819175911 +0000 UTC m=+2142.847734248" Jul 13 19:34:03.573196 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.573156 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s"] Jul 13 19:34:03.576982 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.576955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.584571 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.584543 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-0a1747-serving-cert\"" Jul 13 19:34:03.597478 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.597447 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s"] Jul 13 19:34:03.713460 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.713419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.713663 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.713540 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7x7nx\" (UniqueName: \"kubernetes.io/projected/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-kube-api-access-7x7nx\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.713663 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.713615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.814859 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.814816 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7x7nx\" (UniqueName: \"kubernetes.io/projected/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-kube-api-access-7x7nx\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.815020 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.814890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.815020 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.814931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.815597 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.815573 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.817524 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.817501 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-maas-api-tls\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.833431 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.833404 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7x7nx\" (UniqueName: \"kubernetes.io/projected/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-kube-api-access-7x7nx\") pod \"maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:03.887406 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:03.887363 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:04.040386 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:04.040360 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s"] Jul 13 19:34:04.042259 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:34:04.042238 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee35c90a_83c7_4c5d_a1ff_90a786ead9ad.slice/crio-9d428401062fe826557f522e5be93e8eaf13b0b91b305cea00706c641cc19734 WatchSource:0}: Error finding container 9d428401062fe826557f522e5be93e8eaf13b0b91b305cea00706c641cc19734: Status 404 returned error can't find the container with id 9d428401062fe826557f522e5be93e8eaf13b0b91b305cea00706c641cc19734 Jul 13 19:34:04.809219 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:04.809132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:34:04.831988 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:04.831947 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" event={"ID":"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad","Type":"ContainerStarted","Data":"a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9"} Jul 13 19:34:04.832139 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:04.832007 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" event={"ID":"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad","Type":"ContainerStarted","Data":"9d428401062fe826557f522e5be93e8eaf13b0b91b305cea00706c641cc19734"} Jul 13 19:34:04.832139 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:04.832083 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:34:04.855881 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:04.855818 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" podStartSLOduration=1.351351491 podStartE2EDuration="1.855798946s" totalImagesPullingTime="504.447455ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:34:04.854522049 +0000 UTC m=+2148.883080388" watchObservedRunningTime="2026-07-13 19:34:04.855798946 +0000 UTC m=+2148.884357283" Jul 13 19:34:10.843373 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:34:10.843341 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:35:07.922464 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:07.922421 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm"] Jul 13 19:35:07.923048 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:07.922765 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" podUID="46a5e2a2-6220-462b-bedb-44dd87898a0e" containerName="maas-api" containerID="cri-o://2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977" gracePeriod=30 Jul 13 19:35:08.377294 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.377259 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:35:08.475629 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.475522 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a5e2a2-6220-462b-bedb-44dd87898a0e-maas-api-tls\") pod \"46a5e2a2-6220-462b-bedb-44dd87898a0e\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " Jul 13 19:35:08.475629 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.475591 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbvq5\" (UniqueName: \"kubernetes.io/projected/46a5e2a2-6220-462b-bedb-44dd87898a0e-kube-api-access-fbvq5\") pod \"46a5e2a2-6220-462b-bedb-44dd87898a0e\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " Jul 13 19:35:08.475872 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.475734 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a5e2a2-6220-462b-bedb-44dd87898a0e-openshift-service-ca\") pod \"46a5e2a2-6220-462b-bedb-44dd87898a0e\" (UID: \"46a5e2a2-6220-462b-bedb-44dd87898a0e\") " Jul 13 19:35:08.476119 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.476089 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46a5e2a2-6220-462b-bedb-44dd87898a0e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46a5e2a2-6220-462b-bedb-44dd87898a0e" (UID: "46a5e2a2-6220-462b-bedb-44dd87898a0e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:35:08.477896 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.477865 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46a5e2a2-6220-462b-bedb-44dd87898a0e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46a5e2a2-6220-462b-bedb-44dd87898a0e" (UID: "46a5e2a2-6220-462b-bedb-44dd87898a0e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:35:08.478154 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.478133 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46a5e2a2-6220-462b-bedb-44dd87898a0e-kube-api-access-fbvq5" (OuterVolumeSpecName: "kube-api-access-fbvq5") pod "46a5e2a2-6220-462b-bedb-44dd87898a0e" (UID: "46a5e2a2-6220-462b-bedb-44dd87898a0e"). InnerVolumeSpecName "kube-api-access-fbvq5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:35:08.576695 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.576637 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46a5e2a2-6220-462b-bedb-44dd87898a0e-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:35:08.576695 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.576700 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46a5e2a2-6220-462b-bedb-44dd87898a0e-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:35:08.576902 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:08.576718 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbvq5\" (UniqueName: \"kubernetes.io/projected/46a5e2a2-6220-462b-bedb-44dd87898a0e-kube-api-access-fbvq5\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:35:09.176421 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.176384 2580 generic.go:356] "Generic (PLEG): container finished" podID="46a5e2a2-6220-462b-bedb-44dd87898a0e" containerID="2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977" exitCode=0 Jul 13 19:35:09.176421 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.176424 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" Jul 13 19:35:09.176909 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.176479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" event={"ID":"46a5e2a2-6220-462b-bedb-44dd87898a0e","Type":"ContainerDied","Data":"2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977"} Jul 13 19:35:09.176909 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.176530 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm" event={"ID":"46a5e2a2-6220-462b-bedb-44dd87898a0e","Type":"ContainerDied","Data":"95c9c00d8a39e345e9fb1b58b35a5cb816175e08017ca9b5887b9ce09e06741c"} Jul 13 19:35:09.176909 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.176555 2580 scope.go:117] "RemoveContainer" containerID="2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977" Jul 13 19:35:09.185546 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.185529 2580 scope.go:117] "RemoveContainer" containerID="2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977" Jul 13 19:35:09.185858 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:35:09.185839 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977\": container with ID starting with 2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977 not found: ID does not exist" containerID="2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977" Jul 13 19:35:09.185904 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.185868 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977"} err="failed to get container status \"2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977\": rpc error: code = NotFound desc = could not find container \"2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977\": container with ID starting with 2c7d29e9e3e83bf242a87ce864d2042bd2397a43aa5f50d23b9c6e715c91c977 not found: ID does not exist" Jul 13 19:35:09.198486 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.198449 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm"] Jul 13 19:35:09.202394 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:09.202368 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-d0bb2f-587fdb4866-6bkvm"] Jul 13 19:35:10.547161 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:10.547125 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46a5e2a2-6220-462b-bedb-44dd87898a0e" path="/var/lib/kubelet/pods/46a5e2a2-6220-462b-bedb-44dd87898a0e/volumes" Jul 13 19:35:34.191469 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.191429 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s"] Jul 13 19:35:34.191997 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.191740 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" podUID="ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" containerName="maas-api" containerID="cri-o://a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9" gracePeriod=30 Jul 13 19:35:34.442709 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.442634 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:35:34.510945 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.510908 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-maas-api-tls\") pod \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " Jul 13 19:35:34.511139 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.510956 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-openshift-service-ca\") pod \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " Jul 13 19:35:34.511139 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.510979 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7x7nx\" (UniqueName: \"kubernetes.io/projected/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-kube-api-access-7x7nx\") pod \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\" (UID: \"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad\") " Jul 13 19:35:34.511396 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.511367 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" (UID: "ee35c90a-83c7-4c5d-a1ff-90a786ead9ad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:35:34.513117 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.513094 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" (UID: "ee35c90a-83c7-4c5d-a1ff-90a786ead9ad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:35:34.513195 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.513138 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-kube-api-access-7x7nx" (OuterVolumeSpecName: "kube-api-access-7x7nx") pod "ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" (UID: "ee35c90a-83c7-4c5d-a1ff-90a786ead9ad"). InnerVolumeSpecName "kube-api-access-7x7nx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:35:34.612082 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.612041 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:35:34.612082 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.612079 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:35:34.612286 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:34.612096 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7x7nx\" (UniqueName: \"kubernetes.io/projected/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad-kube-api-access-7x7nx\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:35:35.311875 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.311844 2580 generic.go:356] "Generic (PLEG): container finished" podID="ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" containerID="a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9" exitCode=0 Jul 13 19:35:35.312359 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.311883 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" Jul 13 19:35:35.312359 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.311934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" event={"ID":"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad","Type":"ContainerDied","Data":"a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9"} Jul 13 19:35:35.312359 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.311998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s" event={"ID":"ee35c90a-83c7-4c5d-a1ff-90a786ead9ad","Type":"ContainerDied","Data":"9d428401062fe826557f522e5be93e8eaf13b0b91b305cea00706c641cc19734"} Jul 13 19:35:35.312359 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.312035 2580 scope.go:117] "RemoveContainer" containerID="a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9" Jul 13 19:35:35.323488 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.323465 2580 scope.go:117] "RemoveContainer" containerID="a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9" Jul 13 19:35:35.323946 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:35:35.323919 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9\": container with ID starting with a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9 not found: ID does not exist" containerID="a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9" Jul 13 19:35:35.324015 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.323957 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9"} err="failed to get container status \"a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9\": rpc error: code = NotFound desc = could not find container \"a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9\": container with ID starting with a1adce2519832a6d82d752c3c23ef23a77c87acd596015e074ea8d6d6438a5e9 not found: ID does not exist" Jul 13 19:35:35.329221 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.329194 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s"] Jul 13 19:35:35.331351 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:35.331331 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-0a1747-7b8bf94b67-ckg2s"] Jul 13 19:35:36.546855 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:36.546822 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" path="/var/lib/kubelet/pods/ee35c90a-83c7-4c5d-a1ff-90a786ead9ad/volumes" Jul 13 19:35:53.047859 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.047765 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w"] Jul 13 19:35:53.048387 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.048321 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" containerName="maas-api" Jul 13 19:35:53.048387 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.048335 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" containerName="maas-api" Jul 13 19:35:53.048387 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.048359 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46a5e2a2-6220-462b-bedb-44dd87898a0e" containerName="maas-api" Jul 13 19:35:53.048387 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.048368 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="46a5e2a2-6220-462b-bedb-44dd87898a0e" containerName="maas-api" Jul 13 19:35:53.048545 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.048443 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee35c90a-83c7-4c5d-a1ff-90a786ead9ad" containerName="maas-api" Jul 13 19:35:53.048545 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.048453 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="46a5e2a2-6220-462b-bedb-44dd87898a0e" containerName="maas-api" Jul 13 19:35:53.051611 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.051592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.054234 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.054210 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-868895-serving-cert\"" Jul 13 19:35:53.063003 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.062972 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w"] Jul 13 19:35:53.180472 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.180432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.180682 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.180494 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fcc5\" (UniqueName: \"kubernetes.io/projected/6bfcba86-4340-4d19-a84a-957ea82ad4c8-kube-api-access-5fcc5\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.180682 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.180558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfcba86-4340-4d19-a84a-957ea82ad4c8-openshift-service-ca\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.281524 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.281477 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.281777 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.281554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fcc5\" (UniqueName: \"kubernetes.io/projected/6bfcba86-4340-4d19-a84a-957ea82ad4c8-kube-api-access-5fcc5\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.281777 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.281610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfcba86-4340-4d19-a84a-957ea82ad4c8-openshift-service-ca\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.281777 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:35:53.281642 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-868895-serving-cert: secret "maas-api-e2e-api-a-868895-serving-cert" not found Jul 13 19:35:53.281777 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:35:53.281726 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls podName:6bfcba86-4340-4d19-a84a-957ea82ad4c8 nodeName:}" failed. No retries permitted until 2026-07-13 19:35:53.781704578 +0000 UTC m=+2257.810262913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls") pod "maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" (UID: "6bfcba86-4340-4d19-a84a-957ea82ad4c8") : secret "maas-api-e2e-api-a-868895-serving-cert" not found Jul 13 19:35:53.282314 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.282295 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfcba86-4340-4d19-a84a-957ea82ad4c8-openshift-service-ca\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.293404 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.293372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fcc5\" (UniqueName: \"kubernetes.io/projected/6bfcba86-4340-4d19-a84a-957ea82ad4c8-kube-api-access-5fcc5\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.787518 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:53.787475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:53.787743 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:35:53.787699 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-868895-serving-cert: secret "maas-api-e2e-api-a-868895-serving-cert" not found Jul 13 19:35:53.787817 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:35:53.787805 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls podName:6bfcba86-4340-4d19-a84a-957ea82ad4c8 nodeName:}" failed. No retries permitted until 2026-07-13 19:35:54.787779965 +0000 UTC m=+2258.816338286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls") pod "maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" (UID: "6bfcba86-4340-4d19-a84a-957ea82ad4c8") : secret "maas-api-e2e-api-a-868895-serving-cert" not found Jul 13 19:35:54.796980 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:54.796931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:54.799431 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:54.799407 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") pod \"maas-api-e2e-api-a-868895-7c655f55f5-8dw5w\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:54.865520 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:54.865478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:55.012046 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:55.012018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w"] Jul 13 19:35:55.013697 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:35:55.013651 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bfcba86_4340_4d19_a84a_957ea82ad4c8.slice/crio-d3cfaad836809b166cea29e57b86797f041973b08d5c4d682dbcd6c6d0bc0c6e WatchSource:0}: Error finding container d3cfaad836809b166cea29e57b86797f041973b08d5c4d682dbcd6c6d0bc0c6e: Status 404 returned error can't find the container with id d3cfaad836809b166cea29e57b86797f041973b08d5c4d682dbcd6c6d0bc0c6e Jul 13 19:35:55.015097 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:55.015073 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 19:35:55.422263 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:55.422230 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" event={"ID":"6bfcba86-4340-4d19-a84a-957ea82ad4c8","Type":"ContainerStarted","Data":"d3cfaad836809b166cea29e57b86797f041973b08d5c4d682dbcd6c6d0bc0c6e"} Jul 13 19:35:57.434241 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:57.434206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" event={"ID":"6bfcba86-4340-4d19-a84a-957ea82ad4c8","Type":"ContainerStarted","Data":"3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8"} Jul 13 19:35:57.434635 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:57.434301 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:35:57.453196 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:35:57.453147 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" podStartSLOduration=2.159270161 podStartE2EDuration="4.453130528s" totalImagesPullingTime="2.293860367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:35:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:35:57.451021069 +0000 UTC m=+2261.479579419" watchObservedRunningTime="2026-07-13 19:35:57.453130528 +0000 UTC m=+2261.481688864" Jul 13 19:36:01.155372 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.155332 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms"] Jul 13 19:36:01.159525 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.159493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.161997 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.161967 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f2d389-serving-cert\"" Jul 13 19:36:01.172731 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.172694 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms"] Jul 13 19:36:01.248509 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.248476 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd3f061-5417-4c89-bd7a-3be9cf77f293-openshift-service-ca\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.248509 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.248513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7zb7\" (UniqueName: \"kubernetes.io/projected/1fd3f061-5417-4c89-bd7a-3be9cf77f293-kube-api-access-p7zb7\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.248824 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.248617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.349865 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.349813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd3f061-5417-4c89-bd7a-3be9cf77f293-openshift-service-ca\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.350089 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.349873 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7zb7\" (UniqueName: \"kubernetes.io/projected/1fd3f061-5417-4c89-bd7a-3be9cf77f293-kube-api-access-p7zb7\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.350089 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.349978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.350212 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:36:01.350097 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-serving-cert: secret "maas-api-e2e-api-b-f2d389-serving-cert" not found Jul 13 19:36:01.350212 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:36:01.350171 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls podName:1fd3f061-5417-4c89-bd7a-3be9cf77f293 nodeName:}" failed. No retries permitted until 2026-07-13 19:36:01.850152661 +0000 UTC m=+2265.878710977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls") pod "maas-api-e2e-api-b-f2d389-9cb444794-d8xms" (UID: "1fd3f061-5417-4c89-bd7a-3be9cf77f293") : secret "maas-api-e2e-api-b-f2d389-serving-cert" not found Jul 13 19:36:01.350773 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.350752 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd3f061-5417-4c89-bd7a-3be9cf77f293-openshift-service-ca\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.361065 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.361033 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7zb7\" (UniqueName: \"kubernetes.io/projected/1fd3f061-5417-4c89-bd7a-3be9cf77f293-kube-api-access-p7zb7\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.853328 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.853289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:01.855811 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:01.855777 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls\") pod \"maas-api-e2e-api-b-f2d389-9cb444794-d8xms\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:02.075495 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:02.075446 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:02.225573 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:02.225539 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms"] Jul 13 19:36:02.226489 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:36:02.226464 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd3f061_5417_4c89_bd7a_3be9cf77f293.slice/crio-807a4627a95c17b485c80d48060c9cba19ecba77c6124cd105ae7d20e493a6c3 WatchSource:0}: Error finding container 807a4627a95c17b485c80d48060c9cba19ecba77c6124cd105ae7d20e493a6c3: Status 404 returned error can't find the container with id 807a4627a95c17b485c80d48060c9cba19ecba77c6124cd105ae7d20e493a6c3 Jul 13 19:36:02.461772 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:02.461683 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" event={"ID":"1fd3f061-5417-4c89-bd7a-3be9cf77f293","Type":"ContainerStarted","Data":"807a4627a95c17b485c80d48060c9cba19ecba77c6124cd105ae7d20e493a6c3"} Jul 13 19:36:03.445869 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:03.445838 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:36:03.468876 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:03.468835 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" event={"ID":"1fd3f061-5417-4c89-bd7a-3be9cf77f293","Type":"ContainerStarted","Data":"203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622"} Jul 13 19:36:03.469054 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:03.468973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:03.487744 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:03.487692 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" podStartSLOduration=2.131448846 podStartE2EDuration="2.48765713s" totalImagesPullingTime="356.208284ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:36:03.486273332 +0000 UTC m=+2267.514831746" watchObservedRunningTime="2026-07-13 19:36:03.48765713 +0000 UTC m=+2267.516215524" Jul 13 19:36:09.481712 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:09.481660 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:24.473164 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.473123 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w"] Jul 13 19:36:24.473655 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.473370 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" podUID="6bfcba86-4340-4d19-a84a-957ea82ad4c8" containerName="maas-api" containerID="cri-o://3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8" gracePeriod=30 Jul 13 19:36:24.729487 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.729412 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:36:24.747623 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.747587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfcba86-4340-4d19-a84a-957ea82ad4c8-openshift-service-ca\") pod \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " Jul 13 19:36:24.747914 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.747659 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fcc5\" (UniqueName: \"kubernetes.io/projected/6bfcba86-4340-4d19-a84a-957ea82ad4c8-kube-api-access-5fcc5\") pod \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " Jul 13 19:36:24.747914 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.747877 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") pod \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\" (UID: \"6bfcba86-4340-4d19-a84a-957ea82ad4c8\") " Jul 13 19:36:24.748932 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.748033 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6bfcba86-4340-4d19-a84a-957ea82ad4c8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6bfcba86-4340-4d19-a84a-957ea82ad4c8" (UID: "6bfcba86-4340-4d19-a84a-957ea82ad4c8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:36:24.748932 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.748250 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6bfcba86-4340-4d19-a84a-957ea82ad4c8-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:36:24.750423 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.750313 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bfcba86-4340-4d19-a84a-957ea82ad4c8-kube-api-access-5fcc5" (OuterVolumeSpecName: "kube-api-access-5fcc5") pod "6bfcba86-4340-4d19-a84a-957ea82ad4c8" (UID: "6bfcba86-4340-4d19-a84a-957ea82ad4c8"). InnerVolumeSpecName "kube-api-access-5fcc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:36:24.750645 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.750623 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6bfcba86-4340-4d19-a84a-957ea82ad4c8" (UID: "6bfcba86-4340-4d19-a84a-957ea82ad4c8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:36:24.849613 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.849565 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6bfcba86-4340-4d19-a84a-957ea82ad4c8-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:36:24.849613 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:24.849602 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fcc5\" (UniqueName: \"kubernetes.io/projected/6bfcba86-4340-4d19-a84a-957ea82ad4c8-kube-api-access-5fcc5\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:36:25.591203 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.591165 2580 generic.go:356] "Generic (PLEG): container finished" podID="6bfcba86-4340-4d19-a84a-957ea82ad4c8" containerID="3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8" exitCode=0 Jul 13 19:36:25.591652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.591217 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" Jul 13 19:36:25.591652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.591254 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" event={"ID":"6bfcba86-4340-4d19-a84a-957ea82ad4c8","Type":"ContainerDied","Data":"3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8"} Jul 13 19:36:25.591652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.591311 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w" event={"ID":"6bfcba86-4340-4d19-a84a-957ea82ad4c8","Type":"ContainerDied","Data":"d3cfaad836809b166cea29e57b86797f041973b08d5c4d682dbcd6c6d0bc0c6e"} Jul 13 19:36:25.591652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.591335 2580 scope.go:117] "RemoveContainer" containerID="3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8" Jul 13 19:36:25.601260 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.601236 2580 scope.go:117] "RemoveContainer" containerID="3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8" Jul 13 19:36:25.601606 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:36:25.601583 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8\": container with ID starting with 3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8 not found: ID does not exist" containerID="3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8" Jul 13 19:36:25.601702 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.601619 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8"} err="failed to get container status \"3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8\": rpc error: code = NotFound desc = could not find container \"3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8\": container with ID starting with 3e1c1817994f458cb8ce71face8f314880c03800e7734169fcc9bbafd62dc3d8 not found: ID does not exist" Jul 13 19:36:25.613465 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.613421 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w"] Jul 13 19:36:25.615647 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:25.615624 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-868895-7c655f55f5-8dw5w"] Jul 13 19:36:26.554652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:26.549593 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bfcba86-4340-4d19-a84a-957ea82ad4c8" path="/var/lib/kubelet/pods/6bfcba86-4340-4d19-a84a-957ea82ad4c8/volumes" Jul 13 19:36:50.056822 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.056783 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms"] Jul 13 19:36:50.057426 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.057029 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" podUID="1fd3f061-5417-4c89-bd7a-3be9cf77f293" containerName="maas-api" containerID="cri-o://203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622" gracePeriod=30 Jul 13 19:36:50.100046 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:36:50.100008 2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd3f061_5417_4c89_bd7a_3be9cf77f293.slice/crio-203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622.scope\": RecentStats: unable to find data in memory cache]" Jul 13 19:36:50.310337 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.310273 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:50.385175 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.385138 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls\") pod \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " Jul 13 19:36:50.385331 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.385265 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7zb7\" (UniqueName: \"kubernetes.io/projected/1fd3f061-5417-4c89-bd7a-3be9cf77f293-kube-api-access-p7zb7\") pod \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " Jul 13 19:36:50.385331 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.385297 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd3f061-5417-4c89-bd7a-3be9cf77f293-openshift-service-ca\") pod \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\" (UID: \"1fd3f061-5417-4c89-bd7a-3be9cf77f293\") " Jul 13 19:36:50.385702 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.385647 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fd3f061-5417-4c89-bd7a-3be9cf77f293-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1fd3f061-5417-4c89-bd7a-3be9cf77f293" (UID: "1fd3f061-5417-4c89-bd7a-3be9cf77f293"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:36:50.387293 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.387270 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1fd3f061-5417-4c89-bd7a-3be9cf77f293" (UID: "1fd3f061-5417-4c89-bd7a-3be9cf77f293"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:36:50.387371 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.387339 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fd3f061-5417-4c89-bd7a-3be9cf77f293-kube-api-access-p7zb7" (OuterVolumeSpecName: "kube-api-access-p7zb7") pod "1fd3f061-5417-4c89-bd7a-3be9cf77f293" (UID: "1fd3f061-5417-4c89-bd7a-3be9cf77f293"). InnerVolumeSpecName "kube-api-access-p7zb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:36:50.486073 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.486020 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7zb7\" (UniqueName: \"kubernetes.io/projected/1fd3f061-5417-4c89-bd7a-3be9cf77f293-kube-api-access-p7zb7\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:36:50.486073 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.486067 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fd3f061-5417-4c89-bd7a-3be9cf77f293-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:36:50.486073 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.486077 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fd3f061-5417-4c89-bd7a-3be9cf77f293-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:36:50.730660 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.730621 2580 generic.go:356] "Generic (PLEG): container finished" podID="1fd3f061-5417-4c89-bd7a-3be9cf77f293" containerID="203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622" exitCode=0 Jul 13 19:36:50.730895 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.730688 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" event={"ID":"1fd3f061-5417-4c89-bd7a-3be9cf77f293","Type":"ContainerDied","Data":"203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622"} Jul 13 19:36:50.730895 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.730696 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" Jul 13 19:36:50.730895 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.730770 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms" event={"ID":"1fd3f061-5417-4c89-bd7a-3be9cf77f293","Type":"ContainerDied","Data":"807a4627a95c17b485c80d48060c9cba19ecba77c6124cd105ae7d20e493a6c3"} Jul 13 19:36:50.730895 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.730799 2580 scope.go:117] "RemoveContainer" containerID="203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622" Jul 13 19:36:50.739854 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.739833 2580 scope.go:117] "RemoveContainer" containerID="203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622" Jul 13 19:36:50.740123 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:36:50.740103 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622\": container with ID starting with 203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622 not found: ID does not exist" containerID="203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622" Jul 13 19:36:50.740164 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.740131 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622"} err="failed to get container status \"203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622\": rpc error: code = NotFound desc = could not find container \"203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622\": container with ID starting with 203fef75c4dd0ef7d95c32720f304a0eebfd8cd27d9d816b8aa72842e32d0622 not found: ID does not exist" Jul 13 19:36:50.747321 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.747293 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms"] Jul 13 19:36:50.749500 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:50.749481 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f2d389-9cb444794-d8xms"] Jul 13 19:36:52.546420 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:36:52.546388 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fd3f061-5417-4c89-bd7a-3be9cf77f293" path="/var/lib/kubelet/pods/1fd3f061-5417-4c89-bd7a-3be9cf77f293/volumes" Jul 13 19:37:09.447180 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447089 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn"] Jul 13 19:37:09.447561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447480 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bfcba86-4340-4d19-a84a-957ea82ad4c8" containerName="maas-api" Jul 13 19:37:09.447561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447495 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bfcba86-4340-4d19-a84a-957ea82ad4c8" containerName="maas-api" Jul 13 19:37:09.447561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447518 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fd3f061-5417-4c89-bd7a-3be9cf77f293" containerName="maas-api" Jul 13 19:37:09.447561 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447524 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fd3f061-5417-4c89-bd7a-3be9cf77f293" containerName="maas-api" Jul 13 19:37:09.447720 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447586 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fd3f061-5417-4c89-bd7a-3be9cf77f293" containerName="maas-api" Jul 13 19:37:09.447720 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.447594 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bfcba86-4340-4d19-a84a-957ea82ad4c8" containerName="maas-api" Jul 13 19:37:09.450982 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.450955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.453604 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.453578 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-196ed1-serving-cert\"" Jul 13 19:37:09.457149 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.457114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69336f07-6c27-4e6c-adf5-478af14792cb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.457301 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.457175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69336f07-6c27-4e6c-adf5-478af14792cb-maas-api-tls\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.457301 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.457266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdsx9\" (UniqueName: \"kubernetes.io/projected/69336f07-6c27-4e6c-adf5-478af14792cb-kube-api-access-bdsx9\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.462647 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.462613 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn"] Jul 13 19:37:09.558211 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.558161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69336f07-6c27-4e6c-adf5-478af14792cb-maas-api-tls\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.558433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.558243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdsx9\" (UniqueName: \"kubernetes.io/projected/69336f07-6c27-4e6c-adf5-478af14792cb-kube-api-access-bdsx9\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.558433 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.558314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69336f07-6c27-4e6c-adf5-478af14792cb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.558994 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.558970 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69336f07-6c27-4e6c-adf5-478af14792cb-openshift-service-ca\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.560761 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.560741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69336f07-6c27-4e6c-adf5-478af14792cb-maas-api-tls\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.566511 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.566484 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdsx9\" (UniqueName: \"kubernetes.io/projected/69336f07-6c27-4e6c-adf5-478af14792cb-kube-api-access-bdsx9\") pod \"maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.763649 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.763552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:09.905593 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:09.905568 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn"] Jul 13 19:37:09.907652 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:37:09.907617 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69336f07_6c27_4e6c_adf5_478af14792cb.slice/crio-8acd10de1a1220f26d06c84921992ad282a0a106fad91363a01df5babb54b320 WatchSource:0}: Error finding container 8acd10de1a1220f26d06c84921992ad282a0a106fad91363a01df5babb54b320: Status 404 returned error can't find the container with id 8acd10de1a1220f26d06c84921992ad282a0a106fad91363a01df5babb54b320 Jul 13 19:37:10.843264 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:10.843219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" event={"ID":"69336f07-6c27-4e6c-adf5-478af14792cb","Type":"ContainerStarted","Data":"8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744"} Jul 13 19:37:10.843701 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:10.843287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" event={"ID":"69336f07-6c27-4e6c-adf5-478af14792cb","Type":"ContainerStarted","Data":"8acd10de1a1220f26d06c84921992ad282a0a106fad91363a01df5babb54b320"} Jul 13 19:37:10.843701 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:10.843396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:10.861640 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:10.861586 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" podStartSLOduration=1.428439849 podStartE2EDuration="1.861570132s" totalImagesPullingTime="433.130283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:37:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:37:10.860743094 +0000 UTC m=+2334.889301434" watchObservedRunningTime="2026-07-13 19:37:10.861570132 +0000 UTC m=+2334.890128506" Jul 13 19:37:16.854942 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:16.854909 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:37:17.610717 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.610680 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt"] Jul 13 19:37:17.615851 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.615829 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.618283 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.618264 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-2555bb-serving-cert\"" Jul 13 19:37:17.622532 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.622506 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt"] Jul 13 19:37:17.733771 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.733733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcec4ee-e00f-40e0-9fd0-0d798a836581-maas-api-tls\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.733771 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.733772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlxfr\" (UniqueName: \"kubernetes.io/projected/8dcec4ee-e00f-40e0-9fd0-0d798a836581-kube-api-access-vlxfr\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.733997 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.733864 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcec4ee-e00f-40e0-9fd0-0d798a836581-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.834695 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.834641 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcec4ee-e00f-40e0-9fd0-0d798a836581-maas-api-tls\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.834885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.834716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlxfr\" (UniqueName: \"kubernetes.io/projected/8dcec4ee-e00f-40e0-9fd0-0d798a836581-kube-api-access-vlxfr\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.834885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.834784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcec4ee-e00f-40e0-9fd0-0d798a836581-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.835463 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.835438 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcec4ee-e00f-40e0-9fd0-0d798a836581-openshift-service-ca\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.837122 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.837104 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcec4ee-e00f-40e0-9fd0-0d798a836581-maas-api-tls\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.844371 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.844348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlxfr\" (UniqueName: \"kubernetes.io/projected/8dcec4ee-e00f-40e0-9fd0-0d798a836581-kube-api-access-vlxfr\") pod \"maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:17.929292 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:17.929204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:18.064821 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:18.064798 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt"] Jul 13 19:37:18.066261 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:37:18.066237 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dcec4ee_e00f_40e0_9fd0_0d798a836581.slice/crio-7180f90706ac08340e5bb5d9fbd6159bfcd3bb1b6c7bc29b5d125a89cce89c5c WatchSource:0}: Error finding container 7180f90706ac08340e5bb5d9fbd6159bfcd3bb1b6c7bc29b5d125a89cce89c5c: Status 404 returned error can't find the container with id 7180f90706ac08340e5bb5d9fbd6159bfcd3bb1b6c7bc29b5d125a89cce89c5c Jul 13 19:37:18.889005 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:18.888970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" event={"ID":"8dcec4ee-e00f-40e0-9fd0-0d798a836581","Type":"ContainerStarted","Data":"9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2"} Jul 13 19:37:18.889191 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:18.889028 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" event={"ID":"8dcec4ee-e00f-40e0-9fd0-0d798a836581","Type":"ContainerStarted","Data":"7180f90706ac08340e5bb5d9fbd6159bfcd3bb1b6c7bc29b5d125a89cce89c5c"} Jul 13 19:37:18.889191 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:18.889094 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:37:18.906798 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:18.906746 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" podStartSLOduration=1.469123059 podStartE2EDuration="1.906726519s" totalImagesPullingTime="437.60346ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 19:37:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:37:18.904548646 +0000 UTC m=+2342.933107029" watchObservedRunningTime="2026-07-13 19:37:18.906726519 +0000 UTC m=+2342.935284858" Jul 13 19:37:24.901796 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:37:24.901766 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:38:16.657495 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:38:16.657467 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:38:16.663017 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:38:16.662988 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:38:16.673410 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:38:16.673382 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:38:16.678362 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:38:16.678339 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:43:16.700130 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:16.700039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:43:16.705760 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:16.705727 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:43:16.709425 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:16.709393 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:43:16.716453 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:16.716429 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nmg8g_c0f5b59c-9501-4223-abfd-30dad4473397/ovn-acl-logging/0.log" Jul 13 19:43:54.269992 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.269936 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn"] Jul 13 19:43:54.270724 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.270218 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" podUID="69336f07-6c27-4e6c-adf5-478af14792cb" containerName="maas-api" containerID="cri-o://8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744" gracePeriod=30 Jul 13 19:43:54.718376 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.718351 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:43:54.913532 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.913497 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69336f07-6c27-4e6c-adf5-478af14792cb-maas-api-tls\") pod \"69336f07-6c27-4e6c-adf5-478af14792cb\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " Jul 13 19:43:54.913933 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.913543 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69336f07-6c27-4e6c-adf5-478af14792cb-openshift-service-ca\") pod \"69336f07-6c27-4e6c-adf5-478af14792cb\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " Jul 13 19:43:54.913933 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.913631 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdsx9\" (UniqueName: \"kubernetes.io/projected/69336f07-6c27-4e6c-adf5-478af14792cb-kube-api-access-bdsx9\") pod \"69336f07-6c27-4e6c-adf5-478af14792cb\" (UID: \"69336f07-6c27-4e6c-adf5-478af14792cb\") " Jul 13 19:43:54.914037 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.913948 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69336f07-6c27-4e6c-adf5-478af14792cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "69336f07-6c27-4e6c-adf5-478af14792cb" (UID: "69336f07-6c27-4e6c-adf5-478af14792cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:43:54.915771 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.915736 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69336f07-6c27-4e6c-adf5-478af14792cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "69336f07-6c27-4e6c-adf5-478af14792cb" (UID: "69336f07-6c27-4e6c-adf5-478af14792cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:43:54.915889 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:54.915813 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69336f07-6c27-4e6c-adf5-478af14792cb-kube-api-access-bdsx9" (OuterVolumeSpecName: "kube-api-access-bdsx9") pod "69336f07-6c27-4e6c-adf5-478af14792cb" (UID: "69336f07-6c27-4e6c-adf5-478af14792cb"). InnerVolumeSpecName "kube-api-access-bdsx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:43:55.014766 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.014730 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/69336f07-6c27-4e6c-adf5-478af14792cb-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:43:55.014766 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.014763 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdsx9\" (UniqueName: \"kubernetes.io/projected/69336f07-6c27-4e6c-adf5-478af14792cb-kube-api-access-bdsx9\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:43:55.015005 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.014777 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/69336f07-6c27-4e6c-adf5-478af14792cb-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:43:55.015763 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.015736 2580 generic.go:356] "Generic (PLEG): container finished" podID="69336f07-6c27-4e6c-adf5-478af14792cb" containerID="8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744" exitCode=0 Jul 13 19:43:55.015878 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.015789 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" Jul 13 19:43:55.015878 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.015827 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" event={"ID":"69336f07-6c27-4e6c-adf5-478af14792cb","Type":"ContainerDied","Data":"8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744"} Jul 13 19:43:55.015987 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.015892 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn" event={"ID":"69336f07-6c27-4e6c-adf5-478af14792cb","Type":"ContainerDied","Data":"8acd10de1a1220f26d06c84921992ad282a0a106fad91363a01df5babb54b320"} Jul 13 19:43:55.015987 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.015927 2580 scope.go:117] "RemoveContainer" containerID="8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744" Jul 13 19:43:55.025652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.025637 2580 scope.go:117] "RemoveContainer" containerID="8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744" Jul 13 19:43:55.025932 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:43:55.025914 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744\": container with ID starting with 8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744 not found: ID does not exist" containerID="8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744" Jul 13 19:43:55.025986 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.025940 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744"} err="failed to get container status \"8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744\": rpc error: code = NotFound desc = could not find container \"8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744\": container with ID starting with 8f19cb450e7a50f6cdf7f8f4fa488de252d535f92e019533ff9454a207eaa744 not found: ID does not exist" Jul 13 19:43:55.037734 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.037709 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn"] Jul 13 19:43:55.041097 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:55.041077 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-196ed1-794cf4d8cb-t4xpn"] Jul 13 19:43:56.547029 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:43:56.546989 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69336f07-6c27-4e6c-adf5-478af14792cb" path="/var/lib/kubelet/pods/69336f07-6c27-4e6c-adf5-478af14792cb/volumes" Jul 13 19:44:20.362229 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.362189 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt"] Jul 13 19:44:20.362784 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.362436 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" podUID="8dcec4ee-e00f-40e0-9fd0-0d798a836581" containerName="maas-api" containerID="cri-o://9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2" gracePeriod=30 Jul 13 19:44:20.813749 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.813715 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:44:20.955619 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.955524 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcec4ee-e00f-40e0-9fd0-0d798a836581-maas-api-tls\") pod \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " Jul 13 19:44:20.955619 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.955603 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vlxfr\" (UniqueName: \"kubernetes.io/projected/8dcec4ee-e00f-40e0-9fd0-0d798a836581-kube-api-access-vlxfr\") pod \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " Jul 13 19:44:20.955849 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.955696 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcec4ee-e00f-40e0-9fd0-0d798a836581-openshift-service-ca\") pod \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\" (UID: \"8dcec4ee-e00f-40e0-9fd0-0d798a836581\") " Jul 13 19:44:20.956112 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.956081 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8dcec4ee-e00f-40e0-9fd0-0d798a836581-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8dcec4ee-e00f-40e0-9fd0-0d798a836581" (UID: "8dcec4ee-e00f-40e0-9fd0-0d798a836581"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 19:44:20.957805 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.957768 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8dcec4ee-e00f-40e0-9fd0-0d798a836581-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8dcec4ee-e00f-40e0-9fd0-0d798a836581" (UID: "8dcec4ee-e00f-40e0-9fd0-0d798a836581"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 19:44:20.957912 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:20.957820 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8dcec4ee-e00f-40e0-9fd0-0d798a836581-kube-api-access-vlxfr" (OuterVolumeSpecName: "kube-api-access-vlxfr") pod "8dcec4ee-e00f-40e0-9fd0-0d798a836581" (UID: "8dcec4ee-e00f-40e0-9fd0-0d798a836581"). InnerVolumeSpecName "kube-api-access-vlxfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 19:44:21.056952 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.056907 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8dcec4ee-e00f-40e0-9fd0-0d798a836581-openshift-service-ca\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:44:21.056952 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.056942 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8dcec4ee-e00f-40e0-9fd0-0d798a836581-maas-api-tls\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:44:21.056952 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.056952 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vlxfr\" (UniqueName: \"kubernetes.io/projected/8dcec4ee-e00f-40e0-9fd0-0d798a836581-kube-api-access-vlxfr\") on node \"ip-10-0-133-70.ec2.internal\" DevicePath \"\"" Jul 13 19:44:21.157472 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.157436 2580 generic.go:356] "Generic (PLEG): container finished" podID="8dcec4ee-e00f-40e0-9fd0-0d798a836581" containerID="9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2" exitCode=0 Jul 13 19:44:21.157652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.157491 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" Jul 13 19:44:21.157652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.157512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" event={"ID":"8dcec4ee-e00f-40e0-9fd0-0d798a836581","Type":"ContainerDied","Data":"9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2"} Jul 13 19:44:21.157652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.157565 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt" event={"ID":"8dcec4ee-e00f-40e0-9fd0-0d798a836581","Type":"ContainerDied","Data":"7180f90706ac08340e5bb5d9fbd6159bfcd3bb1b6c7bc29b5d125a89cce89c5c"} Jul 13 19:44:21.157652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.157588 2580 scope.go:117] "RemoveContainer" containerID="9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2" Jul 13 19:44:21.167258 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.167240 2580 scope.go:117] "RemoveContainer" containerID="9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2" Jul 13 19:44:21.167535 ip-10-0-133-70 kubenswrapper[2580]: E0713 19:44:21.167508 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2\": container with ID starting with 9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2 not found: ID does not exist" containerID="9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2" Jul 13 19:44:21.167593 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.167550 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2"} err="failed to get container status \"9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2\": rpc error: code = NotFound desc = could not find container \"9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2\": container with ID starting with 9ad0d00272597b80fe51472bb245e71c89210013ce81859ace287aa0d6cb1be2 not found: ID does not exist" Jul 13 19:44:21.178828 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.178786 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt"] Jul 13 19:44:21.183214 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:21.183185 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-2555bb-6764d68cf6-v5dgt"] Jul 13 19:44:22.547430 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:22.547392 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8dcec4ee-e00f-40e0-9fd0-0d798a836581" path="/var/lib/kubelet/pods/8dcec4ee-e00f-40e0-9fd0-0d798a836581/volumes" Jul 13 19:44:42.233873 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:42.233843 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-tdc7g_3f6fe140-c81e-4182-bc5b-951f92c12708/copy-manifests/0.log" Jul 13 19:44:42.241526 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:42.241500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-tdc7g_3f6fe140-c81e-4182-bc5b-951f92c12708/manager/0.log" Jul 13 19:44:42.600729 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:42.600698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-779d67c848-9hpgg_290a08de-d79d-4db1-97b3-717d202978a2/maas-api/0.log" Jul 13 19:44:45.140363 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.140330 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf_3fea2deb-622b-49d6-b8a0-8fd77608337e/extract/0.log" Jul 13 19:44:45.157595 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.157563 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf_3fea2deb-622b-49d6-b8a0-8fd77608337e/util/0.log" Jul 13 19:44:45.176062 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.176020 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf_3fea2deb-622b-49d6-b8a0-8fd77608337e/pull/0.log" Jul 13 19:44:45.295922 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.295887 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z_71154e7c-f067-4ba4-912f-803d6dbe83b4/util/0.log" Jul 13 19:44:45.302639 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.302617 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z_71154e7c-f067-4ba4-912f-803d6dbe83b4/pull/0.log" Jul 13 19:44:45.309288 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.309266 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z_71154e7c-f067-4ba4-912f-803d6dbe83b4/extract/0.log" Jul 13 19:44:45.420254 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.420175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp_31ed856b-2822-4a8a-ba4a-5347c838c9ec/util/0.log" Jul 13 19:44:45.427863 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.427835 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp_31ed856b-2822-4a8a-ba4a-5347c838c9ec/pull/0.log" Jul 13 19:44:45.434869 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.434843 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp_31ed856b-2822-4a8a-ba4a-5347c838c9ec/extract/0.log" Jul 13 19:44:45.548755 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.548729 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw_84af5016-3ed7-4857-a098-09f1efa27982/util/0.log" Jul 13 19:44:45.556145 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.556120 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw_84af5016-3ed7-4857-a098-09f1efa27982/pull/0.log" Jul 13 19:44:45.562818 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:45.562798 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw_84af5016-3ed7-4857-a098-09f1efa27982/extract/0.log" Jul 13 19:44:46.880695 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:46.880650 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t2sl2_72da3b81-c8c0-4f08-acd9-90a5b9acee25/istio-proxy/0.log" Jul 13 19:44:47.004384 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:47.004352 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p9456_d30480d3-cadf-4dda-ba68-69b131536262/discovery/0.log" Jul 13 19:44:47.476187 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:47.476159 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-sjxjz_4808c5db-8345-415c-b76e-9b2a59339775/payload-pre-processing/0.log" Jul 13 19:44:47.717391 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:47.717358 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57897d6b58-6nnn2_a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7/router/0.log" Jul 13 19:44:48.286153 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:48.286123 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-8nmmk_f19628e6-608f-40be-accd-929ccdbdc16e/main/0.log" Jul 13 19:44:48.397760 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:48.397714 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbv8fl2_3dd65527-fd50-4416-91f3-e6056cb733c6/main/0.log" Jul 13 19:44:48.510276 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:48.510243 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-cb9r4_3635e5de-7a13-4bf5-b750-81e9850ddf85/main/0.log" Jul 13 19:44:55.754602 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:55.754567 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zsgck_8a86f0ea-4e77-4f6a-a5eb-9c2af94cccaa/global-pull-secret-syncer/0.log" Jul 13 19:44:55.935687 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:55.935651 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bkvh6_9c4684cd-b80d-47bd-8fe8-ecc7f2e17cf7/konnectivity-agent/0.log" Jul 13 19:44:56.046370 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:44:56.046281 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-70.ec2.internal_0c05c575dd3a415123dd60cca6ef09a9/haproxy/0.log" Jul 13 19:45:00.148785 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.148756 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf_3fea2deb-622b-49d6-b8a0-8fd77608337e/extract/0.log" Jul 13 19:45:00.185457 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.185430 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf_3fea2deb-622b-49d6-b8a0-8fd77608337e/util/0.log" Jul 13 19:45:00.211085 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.211049 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759rjdkf_3fea2deb-622b-49d6-b8a0-8fd77608337e/pull/0.log" Jul 13 19:45:00.236888 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.236858 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z_71154e7c-f067-4ba4-912f-803d6dbe83b4/extract/0.log" Jul 13 19:45:00.261005 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.260980 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z_71154e7c-f067-4ba4-912f-803d6dbe83b4/util/0.log" Jul 13 19:45:00.291558 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.291530 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0ssr7z_71154e7c-f067-4ba4-912f-803d6dbe83b4/pull/0.log" Jul 13 19:45:00.334073 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.334046 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp_31ed856b-2822-4a8a-ba4a-5347c838c9ec/extract/0.log" Jul 13 19:45:00.367505 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.367478 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp_31ed856b-2822-4a8a-ba4a-5347c838c9ec/util/0.log" Jul 13 19:45:00.397357 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.397324 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73ncbbp_31ed856b-2822-4a8a-ba4a-5347c838c9ec/pull/0.log" Jul 13 19:45:00.428335 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.428249 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw_84af5016-3ed7-4857-a098-09f1efa27982/extract/0.log" Jul 13 19:45:00.449728 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.449700 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw_84af5016-3ed7-4857-a098-09f1efa27982/util/0.log" Jul 13 19:45:00.474482 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:00.474451 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tccsw_84af5016-3ed7-4857-a098-09f1efa27982/pull/0.log" Jul 13 19:45:03.446509 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:03.446475 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-6tnch_e8dac37f-f511-4f56-9746-827c3213809c/cluster-monitoring-operator/0.log" Jul 13 19:45:03.638311 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:03.638282 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2sx7h_5db6382c-38a2-4cfe-a827-cae55c994448/node-exporter/0.log" Jul 13 19:45:03.662908 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:03.662881 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2sx7h_5db6382c-38a2-4cfe-a827-cae55c994448/kube-rbac-proxy/0.log" Jul 13 19:45:03.686819 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:03.686786 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2sx7h_5db6382c-38a2-4cfe-a827-cae55c994448/init-textfile/0.log" Jul 13 19:45:05.756315 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:05.756287 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-8v42n_ac419028-3515-491f-9b60-e72e2292fea6/networking-console-plugin/0.log" Jul 13 19:45:06.389195 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.389170 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/2.log" Jul 13 19:45:06.394414 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.394389 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-kh496_5db7b1e8-f77b-4e79-a3e9-b5fe96d0c56e/console-operator/3.log" Jul 13 19:45:06.647631 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.647535 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq"] Jul 13 19:45:06.648251 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.648228 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69336f07-6c27-4e6c-adf5-478af14792cb" containerName="maas-api" Jul 13 19:45:06.648251 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.648253 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="69336f07-6c27-4e6c-adf5-478af14792cb" containerName="maas-api" Jul 13 19:45:06.648420 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.648271 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8dcec4ee-e00f-40e0-9fd0-0d798a836581" containerName="maas-api" Jul 13 19:45:06.648420 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.648279 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8dcec4ee-e00f-40e0-9fd0-0d798a836581" containerName="maas-api" Jul 13 19:45:06.648420 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.648374 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="69336f07-6c27-4e6c-adf5-478af14792cb" containerName="maas-api" Jul 13 19:45:06.648420 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.648389 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8dcec4ee-e00f-40e0-9fd0-0d798a836581" containerName="maas-api" Jul 13 19:45:06.651652 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.651631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.654132 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.654114 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hts9g\"/\"openshift-service-ca.crt\"" Jul 13 19:45:06.654221 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.654189 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hts9g\"/\"kube-root-ca.crt\"" Jul 13 19:45:06.655000 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.654981 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hts9g\"/\"default-dockercfg-jhzhj\"" Jul 13 19:45:06.662072 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.662048 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq"] Jul 13 19:45:06.754658 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.754620 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-sys\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.754885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.754764 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkrhf\" (UniqueName: \"kubernetes.io/projected/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-kube-api-access-wkrhf\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.754885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.754835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-proc\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.754885 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.754866 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-podres\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.754989 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.754893 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-lib-modules\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.855996 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.855930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-sys\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkrhf\" (UniqueName: \"kubernetes.io/projected/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-kube-api-access-wkrhf\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856066 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-sys\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856106 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-proc\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-podres\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-lib-modules\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-proc\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856287 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-podres\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.856484 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.856341 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-lib-modules\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.864029 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.864000 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkrhf\" (UniqueName: \"kubernetes.io/projected/94dda56f-1aa3-4417-8895-7d5ea7ecabe8-kube-api-access-wkrhf\") pod \"perf-node-gather-daemonset-6d2fq\" (UID: \"94dda56f-1aa3-4417-8895-7d5ea7ecabe8\") " pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:06.962616 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:06.962509 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:07.102499 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:07.102472 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq"] Jul 13 19:45:07.104046 ip-10-0-133-70 kubenswrapper[2580]: W0713 19:45:07.104022 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod94dda56f_1aa3_4417_8895_7d5ea7ecabe8.slice/crio-ec35401bef11a1f71473d840358e2a59511c6b8b66415beb25b78f58d8f7ad86 WatchSource:0}: Error finding container ec35401bef11a1f71473d840358e2a59511c6b8b66415beb25b78f58d8f7ad86: Status 404 returned error can't find the container with id ec35401bef11a1f71473d840358e2a59511c6b8b66415beb25b78f58d8f7ad86 Jul 13 19:45:07.410152 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:07.410117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" event={"ID":"94dda56f-1aa3-4417-8895-7d5ea7ecabe8","Type":"ContainerStarted","Data":"d7a97e08bce815844b914853f04ad729216c426ac838bf1b5316e7dbfbcfc11d"} Jul 13 19:45:07.410432 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:07.410173 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" event={"ID":"94dda56f-1aa3-4417-8895-7d5ea7ecabe8","Type":"ContainerStarted","Data":"ec35401bef11a1f71473d840358e2a59511c6b8b66415beb25b78f58d8f7ad86"} Jul 13 19:45:07.410432 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:07.410228 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:07.429239 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:07.429185 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" podStartSLOduration=1.429166841 podStartE2EDuration="1.429166841s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 19:45:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 19:45:07.426550211 +0000 UTC m=+2811.455108578" watchObservedRunningTime="2026-07-13 19:45:07.429166841 +0000 UTC m=+2811.457725175" Jul 13 19:45:07.658221 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:07.658197 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-7vrr8_861e258d-9123-4ad0-8e74-d3b682a1746f/volume-data-source-validator/0.log" Jul 13 19:45:08.896026 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:08.896000 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vf6p7_54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d/dns/0.log" Jul 13 19:45:08.939015 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:08.938981 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vf6p7_54f2bedf-2189-4e12-86a1-1fdcd0e9ad8d/kube-rbac-proxy/0.log" Jul 13 19:45:08.977822 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:08.977790 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fmnt7_0b4404b1-b307-4b37-b215-b250669247b7/dns-node-resolver/0.log" Jul 13 19:45:09.681900 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:09.681856 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pkh94_fa191c78-a11e-4519-a32b-c036c00c12cf/node-ca/0.log" Jul 13 19:45:10.692360 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:10.692332 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t2sl2_72da3b81-c8c0-4f08-acd9-90a5b9acee25/istio-proxy/0.log" Jul 13 19:45:10.816122 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:10.816091 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p9456_d30480d3-cadf-4dda-ba68-69b131536262/discovery/0.log" Jul 13 19:45:10.991745 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:10.991642 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-sjxjz_4808c5db-8345-415c-b76e-9b2a59339775/payload-pre-processing/0.log" Jul 13 19:45:11.058091 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:11.058066 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57897d6b58-6nnn2_a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7/router/0.log" Jul 13 19:45:11.760837 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:11.760805 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bk62w_23b75ff0-17af-4e4f-be25-be61ce96cfab/serve-healthcheck-canary/0.log" Jul 13 19:45:12.318321 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:12.318286 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-44q2g_fc33eccb-8c33-46d3-8c49-87ce85e050af/insights-operator/0.log" Jul 13 19:45:12.318893 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:12.318876 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-44q2g_fc33eccb-8c33-46d3-8c49-87ce85e050af/insights-operator/1.log" Jul 13 19:45:12.485344 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:12.485320 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7mw8_4b40a891-5b0a-4f46-b993-0b94d4a5a142/kube-rbac-proxy/0.log" Jul 13 19:45:12.509160 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:12.509121 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7mw8_4b40a891-5b0a-4f46-b993-0b94d4a5a142/exporter/0.log" Jul 13 19:45:12.534184 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:12.534134 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x7mw8_4b40a891-5b0a-4f46-b993-0b94d4a5a142/extractor/0.log" Jul 13 19:45:13.425632 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:13.425608 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hts9g/perf-node-gather-daemonset-6d2fq" Jul 13 19:45:15.721806 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:15.721712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-tdc7g_3f6fe140-c81e-4182-bc5b-951f92c12708/manager/0.log" Jul 13 19:45:15.752543 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:15.752517 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-559f5498f-tdc7g_3f6fe140-c81e-4182-bc5b-951f92c12708/copy-manifests/0.log" Jul 13 19:45:17.254378 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:17.254349 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64bdc88f66-554q8_3bf82596-42ad-45e4-b53d-352af8564b3b/manager/0.log" Jul 13 19:45:17.907701 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:17.903334 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t2sl2_72da3b81-c8c0-4f08-acd9-90a5b9acee25/istio-proxy/0.log" Jul 13 19:45:18.017414 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:18.017376 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p9456_d30480d3-cadf-4dda-ba68-69b131536262/discovery/0.log" Jul 13 19:45:18.176963 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:18.176874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-sjxjz_4808c5db-8345-415c-b76e-9b2a59339775/payload-pre-processing/0.log" Jul 13 19:45:18.229045 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:18.229017 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57897d6b58-6nnn2_a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7/router/0.log" Jul 13 19:45:19.559511 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:19.559479 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4t2sl2_72da3b81-c8c0-4f08-acd9-90a5b9acee25/istio-proxy/0.log" Jul 13 19:45:19.677013 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:19.676985 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-p9456_d30480d3-cadf-4dda-ba68-69b131536262/discovery/0.log" Jul 13 19:45:19.827502 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:19.827483 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-sjxjz_4808c5db-8345-415c-b76e-9b2a59339775/payload-pre-processing/0.log" Jul 13 19:45:19.877930 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:19.877904 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-57897d6b58-6nnn2_a625e2f0-529d-4d3e-a59e-ccbaf6a90ce7/router/0.log" Jul 13 19:45:22.667959 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:22.667930 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-q8cj6_e8bc330b-c3a9-40c1-a071-4154d117a9da/migrator/0.log" Jul 13 19:45:22.695389 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:22.695357 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-q8cj6_e8bc330b-c3a9-40c1-a071-4154d117a9da/graceful-termination/0.log" Jul 13 19:45:23.104196 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:23.104167 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-ngr59_eaea0df7-d85f-48b1-b100-dacf528c63b2/kube-storage-version-migrator-operator/1.log" Jul 13 19:45:23.105092 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:23.105071 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-ngr59_eaea0df7-d85f-48b1-b100-dacf528c63b2/kube-storage-version-migrator-operator/0.log" Jul 13 19:45:24.193147 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.193118 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/kube-multus-additional-cni-plugins/0.log" Jul 13 19:45:24.219032 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.219008 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/egress-router-binary-copy/0.log" Jul 13 19:45:24.241041 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.241019 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/cni-plugins/0.log" Jul 13 19:45:24.264131 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.264107 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/bond-cni-plugin/0.log" Jul 13 19:45:24.287451 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.287418 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/routeoverride-cni/0.log" Jul 13 19:45:24.312964 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.312933 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/whereabouts-cni-bincopy/0.log" Jul 13 19:45:24.339291 ip-10-0-133-70 kubenswrapper[2580]: I0713 19:45:24.339263 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8l58j_4061c19f-9533-4feb-bcf2-3f42c037565e/whereabouts-cni/0.log"