Jul 23 13:40:47.847108 ip-10-0-128-225 systemd[1]: Starting Kubernetes Kubelet... Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.268500 2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272918 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272929 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272933 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272936 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272939 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272942 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272945 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:40:48.364581 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272948 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:40:48.338259 ip-10-0-128-225 systemd[1]: Started Kubernetes Kubelet. Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272951 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272953 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272956 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272960 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272964 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272967 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272970 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272973 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272976 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272979 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272982 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272988 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272991 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272996 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.272999 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273002 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273006 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:40:48.367244 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273010 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273013 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273016 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273018 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273021 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273024 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273027 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273029 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273032 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273034 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273037 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273041 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273044 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273048 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273051 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273055 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273057 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273060 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273062 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273065 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:40:48.368350 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273068 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273071 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273074 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273076 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273079 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273081 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273084 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273086 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273089 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273091 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273094 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273097 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273099 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273102 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273105 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273107 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273110 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273112 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273115 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273118 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:40:48.369147 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273120 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273123 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273126 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273129 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273131 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273134 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273137 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273139 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273142 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273145 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273147 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273150 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273152 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273155 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273158 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273161 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273163 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273165 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273168 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273171 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:40:48.369907 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273173 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273176 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273178 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273181 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273183 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273186 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273188 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273191 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273193 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273196 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273198 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273201 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273204 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273207 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273940 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273948 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273951 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273954 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273957 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273962 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:40:48.370506 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273966 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273969 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273973 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273977 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273980 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273983 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273985 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273988 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273990 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273993 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273996 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.273999 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274001 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274004 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274008 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274010 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274013 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274016 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274018 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274021 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:40:48.371456 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274024 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274026 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274030 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274032 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274035 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274038 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274041 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274043 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274046 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274049 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274052 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274055 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274058 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274060 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274064 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274067 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274070 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274072 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274075 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274078 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:40:48.372362 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274080 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274083 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274086 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274088 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274091 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274094 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274098 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274101 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274104 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274108 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274111 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274114 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274116 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274119 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274122 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274124 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274127 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274130 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274132 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:40:48.373641 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274135 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274137 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274140 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274144 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274147 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274150 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274153 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274155 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274158 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274160 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274163 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274165 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274168 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274170 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274173 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274175 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274178 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274181 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274183 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:40:48.374466 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274189 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274191 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274194 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274196 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274199 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274201 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274204 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274206 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274209 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274211 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274214 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274216 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274219 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274221 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274224 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274570 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274575 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274578 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274581 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274584 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:40:48.375098 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274586 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274589 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274592 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274595 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274597 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274600 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274602 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274605 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274608 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274610 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274614 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274618 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274623 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274625 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274628 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274630 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274633 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274636 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274638 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274640 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:40:48.375714 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274643 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274645 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274648 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274651 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274653 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274655 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274658 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274661 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274663 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274666 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274668 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274671 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274674 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274676 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274679 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274681 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274684 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274686 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274689 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:40:48.376317 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274691 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274694 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274696 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274698 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274701 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274704 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274708 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274712 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274716 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274719 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274722 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274724 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274727 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274730 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274732 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274735 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274737 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274740 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274744 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:40:48.377106 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274747 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274749 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274751 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274754 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274757 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274759 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274761 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274764 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274767 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274771 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274774 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274777 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274779 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274782 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274785 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274788 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274790 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274793 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274797 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:40:48.545043 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274800 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274802 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274805 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274807 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274810 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274813 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274815 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274818 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274821 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274823 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274826 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274828 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274831 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274833 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274836 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274838 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.274856 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274933 2582 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274941 2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274948 2582 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274952 2582 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 13:40:48.565702 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274957 2582 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274960 2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274964 2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274968 2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274971 2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274974 2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274977 2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274980 2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274983 2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274986 2582 flags.go:64] FLAG: --cgroup-root="" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274991 2582 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274994 2582 flags.go:64] FLAG: --client-ca-file="" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.274997 2582 flags.go:64] FLAG: --cloud-provider="external" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275000 2582 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275003 2582 flags.go:64] FLAG: --cluster-domain="" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275006 2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275010 2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275013 2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275016 2582 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275020 2582 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275022 2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275025 2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275029 2582 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275032 2582 flags.go:64] FLAG: --contention-profiling="false" Jul 23 13:40:48.566503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275035 2582 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275038 2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275041 2582 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275046 2582 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275050 2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275053 2582 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275056 2582 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275059 2582 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275062 2582 flags.go:64] FLAG: --enable-server="true" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275065 2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275072 2582 flags.go:64] FLAG: --event-burst="100" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275075 2582 flags.go:64] FLAG: --event-qps="50" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275078 2582 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275081 2582 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275083 2582 flags.go:64] FLAG: --eviction-hard="" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275087 2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275090 2582 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275093 2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275097 2582 flags.go:64] FLAG: --eviction-soft="" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275100 2582 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275103 2582 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275106 2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275108 2582 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275111 2582 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275114 2582 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 13:40:48.567328 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275117 2582 flags.go:64] FLAG: --feature-gates="" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275120 2582 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275123 2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275126 2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275129 2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275132 2582 flags.go:64] FLAG: --healthz-port="10248" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275135 2582 flags.go:64] FLAG: --help="false" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275138 2582 flags.go:64] FLAG: --hostname-override="ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275141 2582 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275144 2582 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275148 2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275151 2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275154 2582 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275157 2582 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275160 2582 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275163 2582 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275165 2582 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275169 2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275172 2582 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275174 2582 flags.go:64] FLAG: --kube-reserved="" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275177 2582 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275180 2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275183 2582 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275185 2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 13:40:48.568125 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275188 2582 flags.go:64] FLAG: --lock-file="" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275192 2582 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275195 2582 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275198 2582 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275203 2582 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275206 2582 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275209 2582 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275211 2582 flags.go:64] FLAG: --logging-format="text" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275214 2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275217 2582 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275220 2582 flags.go:64] FLAG: --manifest-url="" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275223 2582 flags.go:64] FLAG: --manifest-url-header="" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275227 2582 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275230 2582 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275234 2582 flags.go:64] FLAG: --max-pods="110" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275236 2582 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275239 2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275242 2582 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275246 2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275249 2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275252 2582 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275255 2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275261 2582 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275264 2582 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275267 2582 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 13:40:48.568916 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275269 2582 flags.go:64] FLAG: --pod-cidr="" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275273 2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275278 2582 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275281 2582 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275284 2582 flags.go:64] FLAG: --pods-per-core="0" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275287 2582 flags.go:64] FLAG: --port="10250" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275290 2582 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275292 2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-001bf3f6be7bfdd4b" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275296 2582 flags.go:64] FLAG: --qos-reserved="" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275300 2582 flags.go:64] FLAG: --read-only-port="10255" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275303 2582 flags.go:64] FLAG: --register-node="true" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275306 2582 flags.go:64] FLAG: --register-with-taints="" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275309 2582 flags.go:64] FLAG: --registry-burst="10" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275312 2582 flags.go:64] FLAG: --registry-qps="5" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275315 2582 flags.go:64] FLAG: --reserved-cpus="" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275318 2582 flags.go:64] FLAG: --reserved-memory="" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275321 2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275324 2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275327 2582 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275330 2582 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275333 2582 flags.go:64] FLAG: --runonce="false" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275335 2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275338 2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275341 2582 flags.go:64] FLAG: --seccomp-default="false" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275344 2582 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 13:40:48.569685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275348 2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275350 2582 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275353 2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275356 2582 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275359 2582 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275362 2582 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275365 2582 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275367 2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275370 2582 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275374 2582 flags.go:64] FLAG: --system-cgroups="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275376 2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275382 2582 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275385 2582 flags.go:64] FLAG: --tls-cert-file="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275387 2582 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275391 2582 flags.go:64] FLAG: --tls-min-version="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275394 2582 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275397 2582 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275400 2582 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275403 2582 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275406 2582 flags.go:64] FLAG: --v="2" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275410 2582 flags.go:64] FLAG: --version="false" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275413 2582 flags.go:64] FLAG: --vmodule="" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275417 2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.275420 2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275537 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:40:48.570675 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275541 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275544 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275547 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275550 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275553 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275555 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275557 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275561 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275564 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275567 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275570 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275572 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275575 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275578 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275580 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275583 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275585 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275588 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275590 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275593 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:40:48.571451 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275596 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275598 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275600 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275604 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275607 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275609 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275612 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275615 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275617 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275620 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275622 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275625 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275627 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275630 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275632 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275635 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275637 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275640 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275642 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275646 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:40:48.710392 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275648 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275651 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275654 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275656 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275659 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275661 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275663 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275666 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275668 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275671 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275674 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275676 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275679 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275682 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275684 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275688 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275690 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275693 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275695 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275698 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:40:48.711590 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275700 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275703 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275706 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275708 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275711 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275713 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275716 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275719 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275721 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275724 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275727 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275732 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275735 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275738 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275741 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275745 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275749 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275752 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275756 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:40:48.712448 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275759 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275762 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275765 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275768 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275776 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275779 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275782 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275785 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275788 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275791 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275795 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275797 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275800 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275802 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275805 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275807 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275810 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275813 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:40:48.713119 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.275815 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.276662 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.284661 2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.284678 2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.285589 2582 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.285602 2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285678 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285684 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285689 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285693 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285697 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285702 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285709 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285715 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:40:48.713657 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285720 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285724 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285729 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285734 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285737 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285741 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285746 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285750 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285754 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285758 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285762 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285766 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285770 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285774 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285778 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285782 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285786 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285790 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285794 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285800 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:40:48.714153 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285804 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285809 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285813 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285817 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285821 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285825 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285829 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285832 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285836 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285854 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285858 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285862 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285865 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285869 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285872 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285876 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285880 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285883 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285887 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285891 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:40:48.714743 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285895 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285898 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285902 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285906 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285910 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285913 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285917 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285921 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285926 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285930 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285934 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285939 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285943 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285947 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285952 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285956 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285959 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285963 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285967 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285970 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:40:48.715385 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285974 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285978 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285982 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285986 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285989 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285993 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.285997 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286002 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286005 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286008 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286014 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286020 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286026 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286030 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286034 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286037 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286041 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286044 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286048 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:40:48.716040 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286053 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286058 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286061 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286065 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286071 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286075 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286079 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286082 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286087 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286091 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286094 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286099 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.286105 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286258 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286265 2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 13:40:48.716635 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286269 2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286274 2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286280 2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286284 2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286288 2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286292 2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286295 2582 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286299 2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286303 2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286307 2582 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286312 2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286315 2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286320 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286323 2582 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286327 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286331 2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286336 2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286341 2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286345 2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 13:40:48.717104 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286351 2582 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286355 2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286361 2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286364 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286368 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286372 2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286376 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286382 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286386 2582 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286390 2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286393 2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286397 2582 feature_gate.go:328] unrecognized feature gate: Example Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286401 2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286405 2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286408 2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286412 2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286417 2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286420 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286424 2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286428 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 13:40:48.717699 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286431 2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286435 2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286439 2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286442 2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286446 2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286450 2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286454 2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286458 2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286461 2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286465 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286468 2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286472 2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286479 2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286483 2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286489 2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286493 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286496 2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286500 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286505 2582 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 13:40:48.718278 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286508 2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286513 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286517 2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286521 2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286525 2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286529 2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286533 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286537 2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286541 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286544 2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286548 2582 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286552 2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286555 2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286559 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286563 2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286566 2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286570 2582 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286574 2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286579 2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286584 2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 13:40:48.718863 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286588 2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286593 2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286597 2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286601 2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286605 2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286610 2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286614 2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286620 2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286624 2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286628 2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286632 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286635 2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286639 2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286642 2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286647 2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286651 2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286655 2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286658 2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 13:40:48.719464 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:48.286663 2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.286670 2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.286795 2582 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.288961 2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.290140 2582 server.go:1019] "Starting client certificate rotation" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.290232 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.290270 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.292718 2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.294167 2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.304634 2582 log.go:25] "Validated CRI v1 runtime API" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.308113 2582 log.go:25] "Validated CRI v1 image API" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.309011 2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 13:40:48.720066 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.311761 2582 fs.go:135] Filesystem UUIDs: map[20deb69d-c4dc-439c-aa47-33db359ef261:/dev/nvme0n1p3 72f4eba5-68d3-4668-ac17-c5a1d514391f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 23 13:40:48.720438 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.311775 2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.316592 2582 manager.go:216] Machine: {Timestamp:2026-07-23 13:40:48.315289017 +0000 UTC m=+0.337000888 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3084106 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f48ce261eb8a023df5502aa6542a6 SystemUUID:ec2f48ce-261e-b8a0-23df-5502aa6542a6 BootID:f383876a-a152-4a6f-a9be-1d74d2c6019d Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:08:2e:73:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:08:2e:73:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:c2:15:4d:62:6b:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.316687 2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.316780 2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.320406 2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.320425 2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-225.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.320639 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 13:40:48.720548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.321698 2582 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.321709 2582 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.321719 2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.322541 2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.323717 2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.326926 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327054 2582 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327084 2582 kubelet.go:486] "Attempting to sync node with API server" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327098 2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327115 2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327127 2582 kubelet.go:398] "Adding apiserver pod source" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327139 2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327491 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.327508 2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.329633 2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.331402 2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.331428 2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333032 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333049 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333054 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333059 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333068 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333074 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333082 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333088 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333099 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333105 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333114 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.333120 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.334333 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.334339 2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.337375 2582 server.go:1268] "Started kubelet" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.337458 2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.337458 2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.337541 2582 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.341728 2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-225.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.343446 2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.344135 2582 server.go:310] "Adding debug handlers to kubelet server" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.344258 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 13:40:48.720858 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.344265 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.344313 2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349306 2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349312 2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.349466 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.348795 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-225.ec2.internal.18c4eeebad465267 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-225.ec2.internal,UID:ip-10-0-128-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-225.ec2.internal,},FirstTimestamp:2026-07-23 13:40:48.337351271 +0000 UTC m=+0.359063142,LastTimestamp:2026-07-23 13:40:48.337351271 +0000 UTC m=+0.359063142,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-225.ec2.internal,}" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349928 2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349928 2582 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349948 2582 factory.go:55] Registering systemd factory Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349954 2582 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.349962 2582 factory.go:223] Registration of the systemd container factory successfully Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350081 2582 reconstruct.go:97] "Volume reconstruction finished" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350088 2582 reconciler.go:29] "Reconciler: start to sync state" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350282 2582 factory.go:153] Registering CRI-O factory Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350290 2582 factory.go:223] Registration of the crio container factory successfully Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350350 2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350376 2582 factory.go:103] Registering Raw factory Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350397 2582 manager.go:1196] Started watching for new ooms in manager Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350816 2582 manager.go:349] Starting recovery of all containers Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.350998 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hr4xf" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.358383 2582 manager.go:354] Recovery completed Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.364555 2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.364652 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hr4xf" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.357813 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.364653 2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-225.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.368388 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.371407 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.371435 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.371449 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372057 2582 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372069 2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372071 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.722023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372086 2582 state_mem.go:36] "Initialized new in-memory state store" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372891 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372921 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.372936 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.373289 2582 policy_none.go:49] "None policy: Start" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.373302 2582 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.373315 2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.373375 2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-225.ec2.internal.18c4eeebaf4e2958 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-225.ec2.internal,UID:ip-10-0-128-225.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-225.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-225.ec2.internal,},FirstTimestamp:2026-07-23 13:40:48.37141948 +0000 UTC m=+0.393131357,LastTimestamp:2026-07-23 13:40:48.37141948 +0000 UTC m=+0.393131357,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-225.ec2.internal,}" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.374481 2582 policy_none.go:47] "Start" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.420531 2582 manager.go:337] "Starting Device Plugin manager" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.420631 2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.420650 2582 server.go:87] "Starting device plugin registration server" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.420893 2582 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.420905 2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.421065 2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.421160 2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.421189 2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.421476 2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.421520 2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.521125 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.521968 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.521995 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.522010 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.522041 2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.535571 2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.535593 2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-225.ec2.internal\": node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.551888 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.557032 2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.557069 2582 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.557102 2582 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.557139 2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.560084 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:40:48.723302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.651935 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.657888 2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal"] Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.657968 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.658941 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.658972 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.658983 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.660563 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.660728 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.660756 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.661412 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.661417 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.661442 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.661454 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.661458 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.661470 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.662611 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.662634 2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.663316 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientMemory" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.663337 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.663349 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeHasSufficientPID" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.688594 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-225.ec2.internal\" not found" node="ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.724631 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.693509 2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-225.ec2.internal\" not found" node="ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.751982 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1cab1d6084d8841ec0ac45f2a8bd856-config\") pod \"kube-apiserver-proxy-ip-10-0-128-225.ec2.internal\" (UID: \"a1cab1d6084d8841ec0ac45f2a8bd856\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.752002 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.752012 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d6a2b36f627916cf37e3614fba833a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal\" (UID: \"7d6a2b36f627916cf37e3614fba833a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.752048 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d6a2b36f627916cf37e3614fba833a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal\" (UID: \"7d6a2b36f627916cf37e3614fba833a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.852913 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.852976 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1cab1d6084d8841ec0ac45f2a8bd856-config\") pod \"kube-apiserver-proxy-ip-10-0-128-225.ec2.internal\" (UID: \"a1cab1d6084d8841ec0ac45f2a8bd856\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.853002 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d6a2b36f627916cf37e3614fba833a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal\" (UID: \"7d6a2b36f627916cf37e3614fba833a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.853031 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d6a2b36f627916cf37e3614fba833a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal\" (UID: \"7d6a2b36f627916cf37e3614fba833a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.853043 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a1cab1d6084d8841ec0ac45f2a8bd856-config\") pod \"kube-apiserver-proxy-ip-10-0-128-225.ec2.internal\" (UID: \"a1cab1d6084d8841ec0ac45f2a8bd856\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.853087 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d6a2b36f627916cf37e3614fba833a6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal\" (UID: \"7d6a2b36f627916cf37e3614fba833a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.880046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.853087 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7d6a2b36f627916cf37e3614fba833a6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal\" (UID: \"7d6a2b36f627916cf37e3614fba833a6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.996799 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:48.953411 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:48.996799 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.994630 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:48.998242 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:48.996928 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" Jul 23 13:40:49.121662 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.054434 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.246039 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.155030 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.256375 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.255441 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.366892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:49.289678 2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 13:40:49.366892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:49.289827 2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 13:40:49.366892 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.356020 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.371232 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:49.370197 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 13:35:48 +0000 UTC" deadline="2028-02-02 20:12:54.122611561 +0000 UTC" Jul 23 13:40:49.371232 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:49.370235 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13422h32m4.752381437s" Jul 23 13:40:49.499364 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.456816 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.629769 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.557399 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.755466 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.657908 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.758843 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.758217 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:49.886691 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.859114 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:50.005658 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:49.959796 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:50.118357 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.060211 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:50.160767 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.160707 2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-225.ec2.internal\" not found" Jul 23 13:40:50.173164 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.173141 2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:40:50.173308 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.173207 2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:40:50.175878 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.175852 2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:40:50.177603 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.177582 2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 13:40:50.249576 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.249544 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" Jul 23 13:40:50.262146 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.262121 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 13:40:50.263393 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.263379 2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" Jul 23 13:40:50.270965 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.270950 2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 13:40:50.303801 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.303770 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d6a2b36f627916cf37e3614fba833a6.slice/crio-edb448dd18f29e83fb6cbb8302bdfc26cf3b67a98228c070cd21987a853599e3 WatchSource:0}: Error finding container edb448dd18f29e83fb6cbb8302bdfc26cf3b67a98228c070cd21987a853599e3: Status 404 returned error can't find the container with id edb448dd18f29e83fb6cbb8302bdfc26cf3b67a98228c070cd21987a853599e3 Jul 23 13:40:50.310079 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.310063 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:40:50.329204 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.329184 2582 apiserver.go:52] "Watching apiserver" Jul 23 13:40:50.335279 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.335263 2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 13:40:50.335655 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.335635 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-6kzs9","kube-system/konnectivity-agent-fr56v","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68","openshift-multus/multus-wqxkn","openshift-network-diagnostics/network-check-target-rc6nl","openshift-network-operator/iptables-alerter-mslmb","openshift-ovn-kubernetes/ovnkube-node-7fwh6","kube-system/global-pull-secret-syncer-rv5cp","kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal","openshift-cluster-node-tuning-operator/tuned-pn4j9","openshift-dns/node-resolver-dmf95","openshift-image-registry/node-ca-s5m98","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal","openshift-multus/multus-additional-cni-plugins-d949x"] Jul 23 13:40:50.338299 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.338283 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.340438 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.340092 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341473 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341568 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341662 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341812 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341955 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341976 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-f6k7q\"" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.341985 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 13:40:50.342769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.342627 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.343096 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.342862 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 13:40:50.343507 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.343483 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-xvl4f\"" Jul 23 13:40:50.343783 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.343655 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.343783 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.343722 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.345145 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.345125 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:50.345243 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.345172 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.345243 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.345193 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:40:50.345365 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.345297 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.345429 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.345414 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 13:40:50.346348 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.346322 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 13:40:50.346500 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.346482 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.346612 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.346598 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.346944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.346815 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-46kdf\"" Jul 23 13:40:50.347070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.347055 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.347124 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.347095 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cns9h\"" Jul 23 13:40:50.347392 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.347376 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 13:40:50.348248 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.348227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.349088 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.349066 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bkvmc\"" Jul 23 13:40:50.349225 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.349204 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.349327 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.349271 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.349327 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.349286 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 13:40:50.349660 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.349644 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.349744 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.349693 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:40:50.349744 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.349732 2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 13:40:50.350505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.350487 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 13:40:50.351121 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.351101 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.351218 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.351125 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 13:40:50.351285 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.351236 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8qxnw\"" Jul 23 13:40:50.352592 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.352576 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.353770 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.353751 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 13:40:50.353872 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.353781 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-btdbd\"" Jul 23 13:40:50.353945 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.353922 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 13:40:50.354225 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.354207 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:50.354308 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.354288 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:40:50.355062 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.355046 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.355157 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.355092 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.355347 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.355332 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gws4w\"" Jul 23 13:40:50.355426 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.355389 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.358422 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.358405 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-m76cg\"" Jul 23 13:40:50.358659 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.358641 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 13:40:50.358930 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.358899 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 13:40:50.359001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.358976 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 13:40:50.361077 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361057 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-hostroot\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361183 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361083 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-system-cni-dir\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.361183 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361100 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-conf-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361183 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361116 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-etc-kubernetes\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361183 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361143 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-os-release\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.361356 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361203 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-cni-netd\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361356 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361242 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-system-cni-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361356 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e7d1c62-d253-4af8-86b5-003a51e94ff9-cni-binary-copy\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361356 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-k8s-cni-cncf-io\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361356 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361303 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-etc-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361356 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361326 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-cni-bin\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361356 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0435abd0-6478-44ee-ba15-68e22040a043-hosts-file\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361415 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-os-release\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361461 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361495 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x6fn\" (UniqueName: \"kubernetes.io/projected/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-kube-api-access-9x6fn\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361521 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-registration-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361543 2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361547 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-log-socket\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361607 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovnkube-config\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-device-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361654 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cfrk\" (UniqueName: \"kubernetes.io/projected/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-kube-api-access-7cfrk\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361679 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-cni-bin\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361698 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-kubelet\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361714 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361732 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-dbus\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361757 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361779 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-cnibin\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361813 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361827 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-cnibin\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361842 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdc090cc-932e-4c22-b4cc-c3cc789fbce0-konnectivity-ca\") pod \"konnectivity-agent-fr56v\" (UID: \"fdc090cc-932e-4c22-b4cc-c3cc789fbce0\") " pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.361944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361909 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-multus-certs\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361962 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435abd0-6478-44ee-ba15-68e22040a043-tmp-dir\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361980 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-node-log\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.361994 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-slash\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362010 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-var-lib-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362041 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362060 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-systemd-units\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362075 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-env-overrides\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362089 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-socket-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362112 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-iptables-alerter-script\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362132 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-kubelet\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362147 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-socket-dir-parent\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qmp2\" (UniqueName: \"kubernetes.io/projected/8e7d1c62-d253-4af8-86b5-003a51e94ff9-kube-api-access-7qmp2\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.362401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362179 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362238 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-systemd\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362262 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6jc8\" (UniqueName: \"kubernetes.io/projected/9a30c045-fceb-4feb-88e0-7e3c610061a8-kube-api-access-t6jc8\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362295 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-host-slash\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362321 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwc6z\" (UniqueName: \"kubernetes.io/projected/0435abd0-6478-44ee-ba15-68e22040a043-kube-api-access-qwc6z\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362344 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swp8j\" (UniqueName: \"kubernetes.io/projected/417b34ab-6072-4887-9455-691c9b0a8bd0-kube-api-access-swp8j\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362368 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-netns\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362391 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-kubelet-config\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362439 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-daemon-config\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362488 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-run-netns\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362515 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-ovn\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362555 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovn-node-metrics-cert\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362584 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-sys-fs\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.363022 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362608 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdc090cc-932e-4c22-b4cc-c3cc789fbce0-agent-certs\") pod \"konnectivity-agent-fr56v\" (UID: \"fdc090cc-932e-4c22-b4cc-c3cc789fbce0\") " pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.363729 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362634 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363729 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362659 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovnkube-script-lib\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.363729 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362706 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.363729 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362744 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-cni-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.363729 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.362768 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-cni-multus\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.363729 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.363618 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.393809 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.393782 2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-485jj" Jul 23 13:40:50.402614 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.402597 2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-485jj" Jul 23 13:40:50.451629 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.451581 2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 13:40:50.464127 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464108 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-systemd-units\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464133 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-env-overrides\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464148 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-socket-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.464195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-iptables-alerter-script\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.464195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464181 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-systemd\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464220 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-tuned\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464249 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bfa6ee4-b6e5-432c-8152-4c008391612b-host\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464249 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-systemd-units\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464280 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-kubelet\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464304 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-socket-dir-parent\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qmp2\" (UniqueName: \"kubernetes.io/projected/8e7d1c62-d253-4af8-86b5-003a51e94ff9-kube-api-access-7qmp2\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464351 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464377 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-kubernetes\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.464398 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464391 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-socket-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464397 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7833f796-95b6-48c1-8be3-b7d78d97215c-tmp\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464481 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-systemd\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464508 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-kubelet\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464556 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464601 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-systemd\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464477 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-socket-dir-parent\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6jc8\" (UniqueName: \"kubernetes.io/projected/9a30c045-fceb-4feb-88e0-7e3c610061a8-kube-api-access-t6jc8\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464707 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-host-slash\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464724 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwc6z\" (UniqueName: \"kubernetes.io/projected/0435abd0-6478-44ee-ba15-68e22040a043-kube-api-access-qwc6z\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464743 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-env-overrides\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464745 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swp8j\" (UniqueName: \"kubernetes.io/projected/417b34ab-6072-4887-9455-691c9b0a8bd0-kube-api-access-swp8j\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464779 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-modprobe-d\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.464861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-sys\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464810 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-netns\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464809 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-iptables-alerter-script\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464827 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-kubelet-config\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464845 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-daemon-config\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464893 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-host-slash\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464928 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-kubelet-config\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464959 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-run-netns\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.464976 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-netns\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-ovn\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovn-node-metrics-cert\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465077 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-sys-fs\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465105 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdc090cc-932e-4c22-b4cc-c3cc789fbce0-agent-certs\") pod \"konnectivity-agent-fr56v\" (UID: \"fdc090cc-932e-4c22-b4cc-c3cc789fbce0\") " pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465151 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-host\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.465601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465153 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465178 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v8rt\" (UniqueName: \"kubernetes.io/projected/7833f796-95b6-48c1-8be3-b7d78d97215c-kube-api-access-9v8rt\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465210 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465269 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovnkube-script-lib\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465292 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465339 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysctl-conf\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465360 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-run-netns\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465426 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-sys-fs\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465440 2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465531 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-daemon-config\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465590 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-run-ovn\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465630 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-cni-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465672 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-cni-multus\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465699 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.466378 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465736 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-hostroot\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465764 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-system-cni-dir\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465794 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-var-lib-kubelet\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465822 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-conf-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465854 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovnkube-script-lib\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465890 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-cni-binary-copy\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-etc-kubernetes\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465936 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-os-release\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465949 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-etc-kubernetes\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465953 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-conf-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465960 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysctl-d\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.465982 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-hostroot\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466009 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-os-release\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466018 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-run\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466035 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-cni-multus\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.467170 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-cni-netd\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466069 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-system-cni-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466078 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-cni-netd\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466091 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e7d1c62-d253-4af8-86b5-003a51e94ff9-cni-binary-copy\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466107 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-multus-cni-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466139 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-system-cni-dir\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466141 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-k8s-cni-cncf-io\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466110 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-k8s-cni-cncf-io\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466159 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-system-cni-dir\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466252 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dd9j\" (UniqueName: \"kubernetes.io/projected/4bfa6ee4-b6e5-432c-8152-4c008391612b-kube-api-access-9dd9j\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466277 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-etc-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466299 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-cni-bin\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466320 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466328 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0435abd0-6478-44ee-ba15-68e22040a043-hosts-file\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466345 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-os-release\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468001 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466365 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-cni-bin\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466427 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466452 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x6fn\" (UniqueName: \"kubernetes.io/projected/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-kube-api-access-9x6fn\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466480 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466519 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wjfk\" (UniqueName: \"kubernetes.io/projected/13853914-f73c-465d-a337-a8bbd8868adb-kube-api-access-2wjfk\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466543 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-registration-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466568 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bfa6ee4-b6e5-432c-8152-4c008391612b-serviceca\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466579 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8e7d1c62-d253-4af8-86b5-003a51e94ff9-cni-binary-copy\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466602 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-log-socket\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466604 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0435abd0-6478-44ee-ba15-68e22040a043-hosts-file\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466638 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-log-socket\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466652 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-os-release\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466658 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-etc-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466700 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovnkube-config\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466707 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-registration-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466753 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-device-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466792 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cfrk\" (UniqueName: \"kubernetes.io/projected/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-kube-api-access-7cfrk\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466805 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-device-dir\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466821 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-cni-bin\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466911 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-kubelet\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466932 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466974 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysconfig\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.466991 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-dbus\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467008 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467033 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467082 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467101 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-cnibin\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.468995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467168 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-cnibin\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467223 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovnkube-config\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467262 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdc090cc-932e-4c22-b4cc-c3cc789fbce0-konnectivity-ca\") pod \"konnectivity-agent-fr56v\" (UID: \"fdc090cc-932e-4c22-b4cc-c3cc789fbce0\") " pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467280 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-kubelet\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467294 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-lib-modules\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467321 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-var-lib-cni-bin\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467325 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-multus-certs\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467356 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435abd0-6478-44ee-ba15-68e22040a043-tmp-dir\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467375 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-dbus\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467418 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-node-log\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467425 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/417b34ab-6072-4887-9455-691c9b0a8bd0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467422 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-host-run-multus-certs\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467457 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-slash\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467482 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-var-lib-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467460 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8e7d1c62-d253-4af8-86b5-003a51e94ff9-cnibin\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.467513 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467518 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.467574 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:40:50.967553051 +0000 UTC m=+2.989264919 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467623 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-etc-selinux\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467652 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-var-lib-openvswitch\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-host-slash\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467713 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0435abd0-6478-44ee-ba15-68e22040a043-tmp-dir\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467702 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/417b34ab-6072-4887-9455-691c9b0a8bd0-cnibin\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.467832 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9a30c045-fceb-4feb-88e0-7e3c610061a8-node-log\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.468245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fdc090cc-932e-4c22-b4cc-c3cc789fbce0-konnectivity-ca\") pod \"konnectivity-agent-fr56v\" (UID: \"fdc090cc-932e-4c22-b4cc-c3cc789fbce0\") " pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.469163 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9a30c045-fceb-4feb-88e0-7e3c610061a8-ovn-node-metrics-cert\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.469966 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.469193 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fdc090cc-932e-4c22-b4cc-c3cc789fbce0-agent-certs\") pod \"konnectivity-agent-fr56v\" (UID: \"fdc090cc-932e-4c22-b4cc-c3cc789fbce0\") " pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.474604 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.474584 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:40:50.474604 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.474607 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:40:50.474783 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.474620 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:50.474783 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.474686 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:40:50.974669393 +0000 UTC m=+2.996381282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:50.475224 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.475194 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swp8j\" (UniqueName: \"kubernetes.io/projected/417b34ab-6072-4887-9455-691c9b0a8bd0-kube-api-access-swp8j\") pod \"multus-additional-cni-plugins-d949x\" (UID: \"417b34ab-6072-4887-9455-691c9b0a8bd0\") " pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.475498 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.475480 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qmp2\" (UniqueName: \"kubernetes.io/projected/8e7d1c62-d253-4af8-86b5-003a51e94ff9-kube-api-access-7qmp2\") pod \"multus-wqxkn\" (UID: \"8e7d1c62-d253-4af8-86b5-003a51e94ff9\") " pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.475593 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.475521 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwc6z\" (UniqueName: \"kubernetes.io/projected/0435abd0-6478-44ee-ba15-68e22040a043-kube-api-access-qwc6z\") pod \"node-resolver-dmf95\" (UID: \"0435abd0-6478-44ee-ba15-68e22040a043\") " pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.476798 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.476780 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6jc8\" (UniqueName: \"kubernetes.io/projected/9a30c045-fceb-4feb-88e0-7e3c610061a8-kube-api-access-t6jc8\") pod \"ovnkube-node-7fwh6\" (UID: \"9a30c045-fceb-4feb-88e0-7e3c610061a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.477045 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.477025 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cfrk\" (UniqueName: \"kubernetes.io/projected/1c6d45d1-7c98-4dd0-8af8-25106ac9958d-kube-api-access-7cfrk\") pod \"aws-ebs-csi-driver-node-gqh68\" (UID: \"1c6d45d1-7c98-4dd0-8af8-25106ac9958d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.477115 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.477078 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x6fn\" (UniqueName: \"kubernetes.io/projected/ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252-kube-api-access-9x6fn\") pod \"iptables-alerter-mslmb\" (UID: \"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252\") " pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.562339 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.562291 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" event={"ID":"7d6a2b36f627916cf37e3614fba833a6","Type":"ContainerStarted","Data":"edb448dd18f29e83fb6cbb8302bdfc26cf3b67a98228c070cd21987a853599e3"} Jul 23 13:40:50.568748 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568725 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-modprobe-d\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568748 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-sys\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568910 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568769 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-host\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568910 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568785 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v8rt\" (UniqueName: \"kubernetes.io/projected/7833f796-95b6-48c1-8be3-b7d78d97215c-kube-api-access-9v8rt\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568910 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568802 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysctl-conf\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568910 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568864 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-sys\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568910 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568883 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-host\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.568910 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568893 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-var-lib-kubelet\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568929 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysctl-d\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-run\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568940 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-modprobe-d\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.568970 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dd9j\" (UniqueName: \"kubernetes.io/projected/4bfa6ee4-b6e5-432c-8152-4c008391612b-kube-api-access-9dd9j\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysctl-conf\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-var-lib-kubelet\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569015 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569030 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-run\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569042 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wjfk\" (UniqueName: \"kubernetes.io/projected/13853914-f73c-465d-a337-a8bbd8868adb-kube-api-access-2wjfk\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569055 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysctl-d\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bfa6ee4-b6e5-432c-8152-4c008391612b-serviceca\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569099 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysconfig\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569154 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-sysconfig\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569171 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569157 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-lib-modules\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569199 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-systemd\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-tuned\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.569157 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569249 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bfa6ee4-b6e5-432c-8152-4c008391612b-host\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569260 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-systemd\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569259 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-lib-modules\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569278 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-kubernetes\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7833f796-95b6-48c1-8be3-b7d78d97215c-tmp\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569311 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-kubernetes\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569320 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bfa6ee4-b6e5-432c-8152-4c008391612b-host\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.569338 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:40:51.069317485 +0000 UTC m=+3.091029353 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:50.569750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.569550 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bfa6ee4-b6e5-432c-8152-4c008391612b-serviceca\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.571327 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.571307 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7833f796-95b6-48c1-8be3-b7d78d97215c-tmp\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.571431 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.571353 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7833f796-95b6-48c1-8be3-b7d78d97215c-etc-tuned\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.577869 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.577849 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dd9j\" (UniqueName: \"kubernetes.io/projected/4bfa6ee4-b6e5-432c-8152-4c008391612b-kube-api-access-9dd9j\") pod \"node-ca-s5m98\" (UID: \"4bfa6ee4-b6e5-432c-8152-4c008391612b\") " pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.578119 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.578094 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v8rt\" (UniqueName: \"kubernetes.io/projected/7833f796-95b6-48c1-8be3-b7d78d97215c-kube-api-access-9v8rt\") pod \"tuned-pn4j9\" (UID: \"7833f796-95b6-48c1-8be3-b7d78d97215c\") " pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.579358 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.579338 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wjfk\" (UniqueName: \"kubernetes.io/projected/13853914-f73c-465d-a337-a8bbd8868adb-kube-api-access-2wjfk\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:50.651776 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.651748 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:40:50.659078 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.659051 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dmf95" Jul 23 13:40:50.660981 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.660956 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a30c045_fceb_4feb_88e0_7e3c610061a8.slice/crio-0dee270b74fe664a17eaea4ab28dbad01474a2ff4518955102e78e3207b94899 WatchSource:0}: Error finding container 0dee270b74fe664a17eaea4ab28dbad01474a2ff4518955102e78e3207b94899: Status 404 returned error can't find the container with id 0dee270b74fe664a17eaea4ab28dbad01474a2ff4518955102e78e3207b94899 Jul 23 13:40:50.667527 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.667512 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" Jul 23 13:40:50.668915 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.668892 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0435abd0_6478_44ee_ba15_68e22040a043.slice/crio-dbdf085fa040c90dd9d0da250547e40eadcc11fdb02b26170fc333e21e921141 WatchSource:0}: Error finding container dbdf085fa040c90dd9d0da250547e40eadcc11fdb02b26170fc333e21e921141: Status 404 returned error can't find the container with id dbdf085fa040c90dd9d0da250547e40eadcc11fdb02b26170fc333e21e921141 Jul 23 13:40:50.673621 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.673601 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wqxkn" Jul 23 13:40:50.675557 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.675487 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c6d45d1_7c98_4dd0_8af8_25106ac9958d.slice/crio-97989f18856a01d56837bb26a9640ab51a36b521966efc53bb2fdc5407354099 WatchSource:0}: Error finding container 97989f18856a01d56837bb26a9640ab51a36b521966efc53bb2fdc5407354099: Status 404 returned error can't find the container with id 97989f18856a01d56837bb26a9640ab51a36b521966efc53bb2fdc5407354099 Jul 23 13:40:50.680042 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.680024 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mslmb" Jul 23 13:40:50.681813 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.681792 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e7d1c62_d253_4af8_86b5_003a51e94ff9.slice/crio-580585c25fa2e43c8bf46245fc2c5a292fd9c51051309bbfc7bf7e52952101bc WatchSource:0}: Error finding container 580585c25fa2e43c8bf46245fc2c5a292fd9c51051309bbfc7bf7e52952101bc: Status 404 returned error can't find the container with id 580585c25fa2e43c8bf46245fc2c5a292fd9c51051309bbfc7bf7e52952101bc Jul 23 13:40:50.685041 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.685005 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d949x" Jul 23 13:40:50.689566 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.689537 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad59f0f5_5ab3_41b0_a5ea_ae7d5619c252.slice/crio-2f4d7c206c376f26e65669f91b5c8f739e3b2f9902c21ee6930d8c0399c4fede WatchSource:0}: Error finding container 2f4d7c206c376f26e65669f91b5c8f739e3b2f9902c21ee6930d8c0399c4fede: Status 404 returned error can't find the container with id 2f4d7c206c376f26e65669f91b5c8f739e3b2f9902c21ee6930d8c0399c4fede Jul 23 13:40:50.690826 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.690805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:40:50.694785 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.694763 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod417b34ab_6072_4887_9455_691c9b0a8bd0.slice/crio-f2bb5f4de152e68bcb22f6ef8a0ed92070a34b9c6f5ece1ade6dad2deae06978 WatchSource:0}: Error finding container f2bb5f4de152e68bcb22f6ef8a0ed92070a34b9c6f5ece1ade6dad2deae06978: Status 404 returned error can't find the container with id f2bb5f4de152e68bcb22f6ef8a0ed92070a34b9c6f5ece1ade6dad2deae06978 Jul 23 13:40:50.696548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.696532 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" Jul 23 13:40:50.698898 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.698879 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdc090cc_932e_4c22_b4cc_c3cc789fbce0.slice/crio-3ef25788100ef3ae3b4abdbb926aaf8acd499cc44038ea81acf94ecf18ac2bbb WatchSource:0}: Error finding container 3ef25788100ef3ae3b4abdbb926aaf8acd499cc44038ea81acf94ecf18ac2bbb: Status 404 returned error can't find the container with id 3ef25788100ef3ae3b4abdbb926aaf8acd499cc44038ea81acf94ecf18ac2bbb Jul 23 13:40:50.701254 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.701238 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-s5m98" Jul 23 13:40:50.704531 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.704511 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7833f796_95b6_48c1_8be3_b7d78d97215c.slice/crio-ba409567cde6b0d88c33b7dfbc27da63f3b5e2249b4ffda46c2a050966eba1fc WatchSource:0}: Error finding container ba409567cde6b0d88c33b7dfbc27da63f3b5e2249b4ffda46c2a050966eba1fc: Status 404 returned error can't find the container with id ba409567cde6b0d88c33b7dfbc27da63f3b5e2249b4ffda46c2a050966eba1fc Jul 23 13:40:50.709528 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:40:50.709509 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bfa6ee4_b6e5_432c_8152_4c008391612b.slice/crio-a0bc28e28cab9a8df76d0651f003ae35ff363eff936f5f0c39fec2ccad641b1c WatchSource:0}: Error finding container a0bc28e28cab9a8df76d0651f003ae35ff363eff936f5f0c39fec2ccad641b1c: Status 404 returned error can't find the container with id a0bc28e28cab9a8df76d0651f003ae35ff363eff936f5f0c39fec2ccad641b1c Jul 23 13:40:50.972214 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:50.972128 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:50.972360 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.972258 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:50.972360 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:50.972322 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:40:51.972305598 +0000 UTC m=+3.994017481 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:51.073553 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.073520 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:51.073553 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.073562 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:51.073802 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.073694 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:51.073802 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.073725 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:40:51.073802 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.073749 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:40:51.073802 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.073755 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:40:52.073740605 +0000 UTC m=+4.095452464 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:51.073802 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.073761 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:51.074178 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.073816 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:40:52.07379924 +0000 UTC m=+4.095511112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:51.404498 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.404434 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 13:35:50 +0000 UTC" deadline="2028-04-11 05:14:29.50105423 +0000 UTC" Jul 23 13:40:51.404498 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.404473 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15063h33m38.096585747s" Jul 23 13:40:51.557769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.557732 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:51.557963 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.557913 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:40:51.580420 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.580328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s5m98" event={"ID":"4bfa6ee4-b6e5-432c-8152-4c008391612b","Type":"ContainerStarted","Data":"a0bc28e28cab9a8df76d0651f003ae35ff363eff936f5f0c39fec2ccad641b1c"} Jul 23 13:40:51.603888 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.603839 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" event={"ID":"7833f796-95b6-48c1-8be3-b7d78d97215c","Type":"ContainerStarted","Data":"ba409567cde6b0d88c33b7dfbc27da63f3b5e2249b4ffda46c2a050966eba1fc"} Jul 23 13:40:51.618907 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.618873 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fr56v" event={"ID":"fdc090cc-932e-4c22-b4cc-c3cc789fbce0","Type":"ContainerStarted","Data":"3ef25788100ef3ae3b4abdbb926aaf8acd499cc44038ea81acf94ecf18ac2bbb"} Jul 23 13:40:51.628297 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.628266 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerStarted","Data":"f2bb5f4de152e68bcb22f6ef8a0ed92070a34b9c6f5ece1ade6dad2deae06978"} Jul 23 13:40:51.644540 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.644473 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqxkn" event={"ID":"8e7d1c62-d253-4af8-86b5-003a51e94ff9","Type":"ContainerStarted","Data":"580585c25fa2e43c8bf46245fc2c5a292fd9c51051309bbfc7bf7e52952101bc"} Jul 23 13:40:51.660994 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.660909 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" event={"ID":"1c6d45d1-7c98-4dd0-8af8-25106ac9958d","Type":"ContainerStarted","Data":"97989f18856a01d56837bb26a9640ab51a36b521966efc53bb2fdc5407354099"} Jul 23 13:40:51.690114 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.690016 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dmf95" event={"ID":"0435abd0-6478-44ee-ba15-68e22040a043","Type":"ContainerStarted","Data":"dbdf085fa040c90dd9d0da250547e40eadcc11fdb02b26170fc333e21e921141"} Jul 23 13:40:51.719856 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.719820 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mslmb" event={"ID":"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252","Type":"ContainerStarted","Data":"2f4d7c206c376f26e65669f91b5c8f739e3b2f9902c21ee6930d8c0399c4fede"} Jul 23 13:40:51.732663 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.732568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"0dee270b74fe664a17eaea4ab28dbad01474a2ff4518955102e78e3207b94899"} Jul 23 13:40:51.982666 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:51.982621 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:51.982858 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.982784 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:51.982858 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:51.982850 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:40:53.982830462 +0000 UTC m=+6.004542329 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:52.083365 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:52.083322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:52.083567 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:52.083383 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:52.083642 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.083570 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:52.083642 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.083639 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:40:54.083618145 +0000 UTC m=+6.105330010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:52.084114 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.084091 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:40:52.084202 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.084120 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:40:52.084202 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.084134 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:52.084202 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.084185 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:40:54.084167654 +0000 UTC m=+6.105879519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:52.405715 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:52.405616 2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 13:35:50 +0000 UTC" deadline="2028-03-30 11:32:27.97780113 +0000 UTC" Jul 23 13:40:52.405715 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:52.405681 2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14781h51m35.572124584s" Jul 23 13:40:52.560783 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:52.560051 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:52.560783 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.560174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:40:52.560783 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:52.560557 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:52.560783 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:52.560649 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:40:53.558130 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:53.558094 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:53.558645 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:53.558245 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:40:54.000275 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:54.000234 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:54.000506 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.000487 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:54.000596 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.000557 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:40:58.000537046 +0000 UTC m=+10.022248927 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:54.101353 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:54.101256 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:54.101353 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:54.101303 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:54.101602 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.101459 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:54.101602 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.101524 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:40:58.101504847 +0000 UTC m=+10.123216708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:54.102120 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.101995 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:40:54.102120 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.102026 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:40:54.102120 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.102040 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:54.102120 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.102093 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:40:58.10207506 +0000 UTC m=+10.123786922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:54.557678 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:54.557631 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:54.557850 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:54.557633 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:54.557850 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.557772 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:40:54.558060 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:54.557880 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:40:55.557748 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:55.557698 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:55.558234 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:55.557863 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:40:56.558260 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:56.558178 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:56.558824 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:56.558317 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:40:56.558824 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:56.558383 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:56.558824 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:56.558480 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:40:57.557692 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:57.557657 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:57.557953 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:57.557806 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:40:58.034950 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:58.034819 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:58.035435 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.034970 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:58.035435 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.035049 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:41:06.035025385 +0000 UTC m=+18.056737245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:40:58.135362 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:58.135323 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:58.135362 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:58.135363 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:58.135569 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.135493 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:58.135569 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.135543 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:41:06.13553028 +0000 UTC m=+18.157242141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:40:58.135875 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.135861 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:40:58.135917 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.135879 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:40:58.135917 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.135888 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:58.135987 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.135920 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:41:06.135909538 +0000 UTC m=+18.157621408 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:40:58.559694 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:58.559062 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:40:58.559694 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.559176 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:40:58.559694 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:58.559543 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:40:58.559694 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:58.559637 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:40:59.558269 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:40:59.558227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:40:59.558786 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:40:59.558378 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:00.557928 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:00.557896 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:00.558116 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:00.558033 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:00.558116 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:00.558087 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:00.558265 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:00.558239 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:01.558286 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:01.558251 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:01.558684 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:01.558401 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:02.557306 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:02.557270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:02.557476 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:02.557411 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:02.557476 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:02.557464 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:02.557608 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:02.557580 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:03.558216 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:03.558182 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:03.558646 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:03.558310 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:04.557541 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:04.557509 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:04.557725 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:04.557509 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:04.557725 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:04.557638 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:04.557824 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:04.557718 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:05.557931 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:05.557883 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:05.558363 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:05.558026 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:06.095486 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:06.095451 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:06.095701 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.095587 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:41:06.095701 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.095652 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:41:22.095634516 +0000 UTC m=+34.117346378 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:41:06.196565 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:06.196516 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:06.196565 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:06.196561 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:06.196807 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.196679 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:41:06.196807 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.196686 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:41:06.196807 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.196699 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:41:06.196807 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.196711 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:41:06.196807 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.196744 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:41:22.196726812 +0000 UTC m=+34.218438680 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:41:06.196807 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.196764 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:41:22.196753143 +0000 UTC m=+34.218465001 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:41:06.558384 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:06.558278 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:06.558806 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.558426 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:06.558806 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:06.558488 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:06.558806 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:06.558593 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:07.557644 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:07.557606 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:07.557833 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:07.557749 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:08.558562 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:08.558535 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:08.559021 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:08.558638 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:08.559021 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:08.558712 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:08.559021 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:08.558800 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:09.557919 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:09.557899 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:09.558013 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:09.557995 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:09.790256 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:09.790206 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1cab1d6084d8841ec0ac45f2a8bd856.slice/crio-4ddd05eb0d2f0ab4e7f032fdd337a25e2e125103c01c497122ede3a91e1ef1e0 WatchSource:0}: Error finding container 4ddd05eb0d2f0ab4e7f032fdd337a25e2e125103c01c497122ede3a91e1ef1e0: Status 404 returned error can't find the container with id 4ddd05eb0d2f0ab4e7f032fdd337a25e2e125103c01c497122ede3a91e1ef1e0 Jul 23 13:41:10.558326 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.557686 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:10.558326 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.557781 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:10.558326 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:10.558043 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:10.558326 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:10.558169 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:10.782574 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.782250 2582 generic.go:356] "Generic (PLEG): container finished" podID="7d6a2b36f627916cf37e3614fba833a6" containerID="22265157f746baf648edda8e673dc0203f07c29593535052186c7d5b3d2b87bb" exitCode=0 Jul 23 13:41:10.782574 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.782427 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" event={"ID":"7d6a2b36f627916cf37e3614fba833a6","Type":"ContainerDied","Data":"22265157f746baf648edda8e673dc0203f07c29593535052186c7d5b3d2b87bb"} Jul 23 13:41:10.786331 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.785645 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-s5m98" event={"ID":"4bfa6ee4-b6e5-432c-8152-4c008391612b","Type":"ContainerStarted","Data":"3ea3c67c85fa72295d266cc1cbe0a14b86755b3243509effdff942c26f5ef68f"} Jul 23 13:41:10.788832 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.788810 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" event={"ID":"7833f796-95b6-48c1-8be3-b7d78d97215c","Type":"ContainerStarted","Data":"6ced92479bd8a9d8f103237b3fda7ab6503dc2c2e4ed1c905758c002a64c948e"} Jul 23 13:41:10.790699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.790670 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fr56v" event={"ID":"fdc090cc-932e-4c22-b4cc-c3cc789fbce0","Type":"ContainerStarted","Data":"0695f04afbb5b60a1c5050e3203fa4fcd1468cb4f83f11082e81163a420c63b6"} Jul 23 13:41:10.793457 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.793421 2582 generic.go:356] "Generic (PLEG): container finished" podID="417b34ab-6072-4887-9455-691c9b0a8bd0" containerID="1528c606f32611607eab83a64fb3c9e7232f819d9c803d73b208beaf6e08b4a5" exitCode=0 Jul 23 13:41:10.793554 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.793502 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerDied","Data":"1528c606f32611607eab83a64fb3c9e7232f819d9c803d73b208beaf6e08b4a5"} Jul 23 13:41:10.799718 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.795780 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wqxkn" event={"ID":"8e7d1c62-d253-4af8-86b5-003a51e94ff9","Type":"ContainerStarted","Data":"6cc10c87d7460a8ba336dcfca1b9af90d12aef6e4898bb467aecdb885d91c726"} Jul 23 13:41:10.801788 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.801662 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" event={"ID":"1c6d45d1-7c98-4dd0-8af8-25106ac9958d","Type":"ContainerStarted","Data":"4bd89933dfe9c6957da9d31027db78174523b27bba49f4138cec787d8bfcda5d"} Jul 23 13:41:10.804233 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.804017 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dmf95" event={"ID":"0435abd0-6478-44ee-ba15-68e22040a043","Type":"ContainerStarted","Data":"d09a60800349b81f8e713a5e9dfee6c0310230e2226dc8d386a80d457089f838"} Jul 23 13:41:10.807072 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.807050 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:41:10.807501 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.807482 2582 generic.go:356] "Generic (PLEG): container finished" podID="9a30c045-fceb-4feb-88e0-7e3c610061a8" containerID="f5f7bca499d6525e25dbce7f507723b7479203020037aa6f55072897240c2789" exitCode=1 Jul 23 13:41:10.807586 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.807551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"e85c91906f28ed38f875988ed1ef8e8b87855f86e57a40b3c94a25560054047c"} Jul 23 13:41:10.807652 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.807614 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"c61e5f69308f4f3226ec688871b8b7e1e54be29664abf3e413fcf080945a2533"} Jul 23 13:41:10.807704 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.807668 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerDied","Data":"f5f7bca499d6525e25dbce7f507723b7479203020037aa6f55072897240c2789"} Jul 23 13:41:10.807754 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.807718 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"c1e4b1b5b4b29657a4316144a7f54b35d0430781d04c1279c3a403183676888e"} Jul 23 13:41:10.809136 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.809116 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" event={"ID":"a1cab1d6084d8841ec0ac45f2a8bd856","Type":"ContainerStarted","Data":"4ddd05eb0d2f0ab4e7f032fdd337a25e2e125103c01c497122ede3a91e1ef1e0"} Jul 23 13:41:10.849978 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.849924 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-s5m98" podStartSLOduration=2.7787897839999998 podStartE2EDuration="21.849905252s" totalImagesPullingTime="19.071115468s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:10.849315701 +0000 UTC m=+22.871027604" watchObservedRunningTime="2026-07-23 13:41:10.849905252 +0000 UTC m=+22.871617134" Jul 23 13:41:10.850165 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.850142 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fr56v" podStartSLOduration=2.84942386 podStartE2EDuration="21.850135056s" totalImagesPullingTime="19.000711196s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:10.835033463 +0000 UTC m=+22.856745343" watchObservedRunningTime="2026-07-23 13:41:10.850135056 +0000 UTC m=+22.871846934" Jul 23 13:41:10.885914 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.885875 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wqxkn" podStartSLOduration=2.7519537290000002 podStartE2EDuration="21.885828892s" totalImagesPullingTime="19.133875163s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:10.867391343 +0000 UTC m=+22.889103222" watchObservedRunningTime="2026-07-23 13:41:10.885828892 +0000 UTC m=+22.907540771" Jul 23 13:41:10.886206 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.886181 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pn4j9" podStartSLOduration=2.811068475 podStartE2EDuration="21.886175487s" totalImagesPullingTime="19.075107012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:10.885734468 +0000 UTC m=+22.907446373" watchObservedRunningTime="2026-07-23 13:41:10.886175487 +0000 UTC m=+22.907887366" Jul 23 13:41:10.906498 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:10.906460 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dmf95" podStartSLOduration=2.796780055 podStartE2EDuration="21.906446743s" totalImagesPullingTime="19.109666688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:10.906324517 +0000 UTC m=+22.928036395" watchObservedRunningTime="2026-07-23 13:41:10.906446743 +0000 UTC m=+22.928158622" Jul 23 13:41:11.558148 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.557915 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:11.558404 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:11.558210 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:11.812772 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.812688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mslmb" event={"ID":"ad59f0f5-5ab3-41b0-a5ea-ae7d5619c252","Type":"ContainerStarted","Data":"0eba34a6b0424ef65c82f35c8b9bb0d16d6d259f39d3b0d6ae74c01f960771c1"} Jul 23 13:41:11.816024 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.815992 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:41:11.816600 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.816573 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"f70259f8c882ee598c5a2913e38b0708113bc37daa77a46f0b0eb985f7cecf34"} Jul 23 13:41:11.816693 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.816659 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"b0342d1ad1acbeea3dabc59663869e5b749fae7308dc7da86706d02d32455769"} Jul 23 13:41:11.818726 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.818671 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" event={"ID":"7d6a2b36f627916cf37e3614fba833a6","Type":"ContainerStarted","Data":"c969e1da33221915ac22911ebe011eabad003657fbf11ff80bdad8b2a89124b2"} Jul 23 13:41:11.832145 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.832096 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mslmb" podStartSLOduration=3.821786244 podStartE2EDuration="22.832081215s" totalImagesPullingTime="19.010294971s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:11.831965659 +0000 UTC m=+23.853677539" watchObservedRunningTime="2026-07-23 13:41:11.832081215 +0000 UTC m=+23.853793095" Jul 23 13:41:11.847186 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:11.847037 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-225.ec2.internal" podStartSLOduration=21.847020468 podStartE2EDuration="21.847020468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:11.846634068 +0000 UTC m=+23.868345947" watchObservedRunningTime="2026-07-23 13:41:11.847020468 +0000 UTC m=+23.868732348" Jul 23 13:41:12.126747 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.126709 2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 13:41:12.434688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.434571 2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T13:41:12.12673217Z","UUID":"2adf7c00-20db-49a0-8774-395d0e460070","Handler":null,"Name":"","Endpoint":""} Jul 23 13:41:12.436330 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.436299 2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 13:41:12.436466 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.436343 2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 13:41:12.557953 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.557923 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:12.557953 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.557938 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:12.558194 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:12.558058 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:12.558409 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:12.558364 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:12.822905 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.822808 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" event={"ID":"a1cab1d6084d8841ec0ac45f2a8bd856","Type":"ContainerStarted","Data":"b62e99b1236ddea9810dc5a1245ab677f64b115642355e55950b618e68b20a44"} Jul 23 13:41:12.824929 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.824899 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" event={"ID":"1c6d45d1-7c98-4dd0-8af8-25106ac9958d","Type":"ContainerStarted","Data":"b06f2ec7dc1b8b338742baa53476ebc96ef7560433c3fff322e089660add807f"} Jul 23 13:41:12.844901 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:12.844865 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-225.ec2.internal" podStartSLOduration=22.844834202 podStartE2EDuration="22.844834202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:12.844053462 +0000 UTC m=+24.865765340" watchObservedRunningTime="2026-07-23 13:41:12.844834202 +0000 UTC m=+24.866546080" Jul 23 13:41:13.558048 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.558029 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:13.558175 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:13.558140 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:13.682820 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.682748 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:41:13.683438 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.683419 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:41:13.828868 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.828812 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" event={"ID":"1c6d45d1-7c98-4dd0-8af8-25106ac9958d","Type":"ContainerStarted","Data":"ec46c9be1f8bb42beb61345911f7d98cf20cd2fd84ac7823423a512599be904d"} Jul 23 13:41:13.832318 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.832297 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:41:13.832685 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.832657 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"5fa98d6099c3170cbb302da2d32cbcc07b781006571f9c539618617f2e87ff1d"} Jul 23 13:41:13.847789 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:13.847744 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gqh68" podStartSLOduration=2.918208721 podStartE2EDuration="24.847729587s" totalImagesPullingTime="21.929520866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:13.847236081 +0000 UTC m=+25.868947961" watchObservedRunningTime="2026-07-23 13:41:13.847729587 +0000 UTC m=+25.869441466" Jul 23 13:41:14.557978 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:14.557943 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:14.558235 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:14.558062 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:14.558235 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:14.558114 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:14.558335 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:14.558265 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:15.373986 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.373884 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dmf95_0435abd0-6478-44ee-ba15-68e22040a043/dns-node-resolver/0.log" Jul 23 13:41:15.557526 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.557500 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:15.557654 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:15.557597 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:15.839279 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.839209 2582 generic.go:356] "Generic (PLEG): container finished" podID="417b34ab-6072-4887-9455-691c9b0a8bd0" containerID="e07c0fdad7d505463a62af59bab88b84020cc7cef09345bbbd3f655fe745a95d" exitCode=0 Jul 23 13:41:15.839411 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.839294 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerDied","Data":"e07c0fdad7d505463a62af59bab88b84020cc7cef09345bbbd3f655fe745a95d"} Jul 23 13:41:15.842769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.842755 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:41:15.843100 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.843079 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"07f88f87beab6cf4c558f3c25c5d5158054c0766a224f02c39b991e6cabaaf92"} Jul 23 13:41:15.843311 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.843295 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:41:15.843365 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.843354 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:41:15.843515 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.843476 2582 scope.go:117] "RemoveContainer" containerID="f5f7bca499d6525e25dbce7f507723b7479203020037aa6f55072897240c2789" Jul 23 13:41:15.857938 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:15.857920 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:41:16.549019 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.548924 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s5m98_4bfa6ee4-b6e5-432c-8152-4c008391612b/node-ca/0.log" Jul 23 13:41:16.557989 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.557958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:16.558142 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.557958 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:16.558142 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:16.558094 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:16.558270 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:16.558160 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:16.599714 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.599681 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:41:16.599873 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.599812 2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 13:41:16.600303 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.600281 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fr56v" Jul 23 13:41:16.848648 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.848584 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:41:16.848989 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.848970 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" event={"ID":"9a30c045-fceb-4feb-88e0-7e3c610061a8","Type":"ContainerStarted","Data":"123674a4f32342d23291c52fde74cf215795ad23ce35005f41e2abfc4fe6d7e4"} Jul 23 13:41:16.849181 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.849155 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:41:16.863488 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.863458 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:41:16.877380 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:16.877338 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" podStartSLOduration=8.335945621 podStartE2EDuration="27.877324211s" totalImagesPullingTime="19.54137859s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:16.876756168 +0000 UTC m=+28.898468048" watchObservedRunningTime="2026-07-23 13:41:16.877324211 +0000 UTC m=+28.899036131" Jul 23 13:41:17.558427 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.558238 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:17.558834 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:17.558529 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:17.641562 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.641523 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rc6nl"] Jul 23 13:41:17.641737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.641627 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:17.641737 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:17.641713 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:17.645014 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.644990 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rv5cp"] Jul 23 13:41:17.645152 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.645077 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:17.645214 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:17.645153 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:17.645557 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.645535 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6kzs9"] Jul 23 13:41:17.853739 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.853654 2582 generic.go:356] "Generic (PLEG): container finished" podID="417b34ab-6072-4887-9455-691c9b0a8bd0" containerID="7fa02ba576d014ed0d95bda0981aeb582e85d82a9f12b98cd2063d54ec1395bd" exitCode=0 Jul 23 13:41:17.853924 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.853734 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerDied","Data":"7fa02ba576d014ed0d95bda0981aeb582e85d82a9f12b98cd2063d54ec1395bd"} Jul 23 13:41:17.853924 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:17.853770 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:17.853924 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:17.853873 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:19.557983 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:19.557945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:19.558397 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:19.557945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:19.558397 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:19.557945 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:19.558397 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:19.558150 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:19.558397 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:19.558056 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:19.558397 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:19.558241 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:21.557725 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:21.557687 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:21.558176 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:21.557811 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:21.558176 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:21.557821 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:21.558176 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:21.557838 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:21.558176 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:21.557929 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:21.558176 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:21.558015 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:22.117886 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:22.117832 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:22.118089 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.118005 2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 13:41:22.118089 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.118077 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret podName:2618d6b8-b672-4af6-bc7d-5e13639fc6a3 nodeName:}" failed. No retries permitted until 2026-07-23 13:41:54.118058212 +0000 UTC m=+66.139770090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret") pod "global-pull-secret-syncer-rv5cp" (UID: "2618d6b8-b672-4af6-bc7d-5e13639fc6a3") : object "kube-system"/"original-pull-secret" not registered Jul 23 13:41:22.218339 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:22.218309 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:22.218339 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:22.218343 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:22.218526 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.218447 2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:41:22.218526 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.218472 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 13:41:22.218526 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.218494 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs podName:13853914-f73c-465d-a337-a8bbd8868adb nodeName:}" failed. No retries permitted until 2026-07-23 13:41:54.218481875 +0000 UTC m=+66.240193735 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs") pod "network-metrics-daemon-6kzs9" (UID: "13853914-f73c-465d-a337-a8bbd8868adb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 13:41:22.218526 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.218496 2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 13:41:22.218526 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.218511 2582 projected.go:196] Error preparing data for projected volume kube-api-access-nnbpv for pod openshift-network-diagnostics/network-check-target-rc6nl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:41:22.218684 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:22.218554 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv podName:e08e055d-d7bb-4173-b10d-6a36561f085c nodeName:}" failed. No retries permitted until 2026-07-23 13:41:54.218541497 +0000 UTC m=+66.240253353 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nnbpv" (UniqueName: "kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv") pod "network-check-target-rc6nl" (UID: "e08e055d-d7bb-4173-b10d-6a36561f085c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 13:41:23.557832 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:23.557803 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:23.557832 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:23.557832 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:23.558349 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:23.557803 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:23.558349 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:23.557934 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:23.558349 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:23.558035 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:23.558349 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:23.558174 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:23.868572 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:23.868538 2582 generic.go:356] "Generic (PLEG): container finished" podID="417b34ab-6072-4887-9455-691c9b0a8bd0" containerID="418b3e4dcc36e91c95af2a45fd08a7d8abbfe9d2b2bd74a8f5cf19f3f993e6d4" exitCode=0 Jul 23 13:41:23.868687 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:23.868607 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerDied","Data":"418b3e4dcc36e91c95af2a45fd08a7d8abbfe9d2b2bd74a8f5cf19f3f993e6d4"} Jul 23 13:41:25.557809 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:25.557605 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:25.558257 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:25.557662 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:25.558257 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:25.557922 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:25.558257 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:25.557659 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:25.558257 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:25.557970 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:25.558257 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:25.558079 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:27.557787 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:27.557742 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:27.557787 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:27.557782 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:27.558271 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:27.557880 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:27.558271 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:27.557955 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:27.558271 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:27.557957 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:27.558271 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:27.558037 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:28.881419 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:28.881380 2582 generic.go:356] "Generic (PLEG): container finished" podID="417b34ab-6072-4887-9455-691c9b0a8bd0" containerID="50147a1c7a3dc25db57aa4753161236ea57c8a080b3e8de16e8162dbfc9a022f" exitCode=0 Jul 23 13:41:28.881799 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:28.881449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerDied","Data":"50147a1c7a3dc25db57aa4753161236ea57c8a080b3e8de16e8162dbfc9a022f"} Jul 23 13:41:29.558307 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:29.558279 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:29.558307 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:29.558304 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:29.558530 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:29.558375 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:29.558530 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:29.558389 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:29.558530 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:29.558443 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:29.558629 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:29.558522 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:29.886629 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:29.886599 2582 generic.go:356] "Generic (PLEG): container finished" podID="417b34ab-6072-4887-9455-691c9b0a8bd0" containerID="e9fcb06324b5bc46d2942f5bc095c2ff69a07b0a5cc1058e93d8d99d4332bd46" exitCode=0 Jul 23 13:41:29.886997 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:29.886637 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerDied","Data":"e9fcb06324b5bc46d2942f5bc095c2ff69a07b0a5cc1058e93d8d99d4332bd46"} Jul 23 13:41:30.891379 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:30.891346 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d949x" event={"ID":"417b34ab-6072-4887-9455-691c9b0a8bd0","Type":"ContainerStarted","Data":"e9caf3a55fe2c2f7b4360b8943b7c982688775f70a2431aacab69b98dea5041a"} Jul 23 13:41:30.915318 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:30.915277 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d949x" podStartSLOduration=6.904685593 podStartE2EDuration="41.91526443s" totalImagesPullingTime="35.010578837s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:30.914810741 +0000 UTC m=+42.936522628" watchObservedRunningTime="2026-07-23 13:41:30.91526443 +0000 UTC m=+42.936976309" Jul 23 13:41:31.557616 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:31.557580 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:31.557814 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:31.557581 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:31.557814 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:31.557696 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:31.557814 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:31.557723 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:31.557814 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:31.557789 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:31.557975 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:31.557889 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:33.557590 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:33.557561 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:33.558042 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:33.557569 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:33.558042 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:33.557658 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:33.558042 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:33.557569 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:33.558042 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:33.557741 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:33.558042 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:33.557797 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:35.558172 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:35.558143 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:35.558537 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:35.558144 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:35.558537 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:35.558249 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:35.558537 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:35.558144 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:35.558537 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:35.558371 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:35.558537 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:35.558312 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:37.557984 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.557959 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:37.558354 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.558054 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:37.558354 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.558071 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:37.558354 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:37.558054 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rv5cp" podUID="2618d6b8-b672-4af6-bc7d-5e13639fc6a3" Jul 23 13:41:37.558354 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:37.558113 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rc6nl" podUID="e08e055d-d7bb-4173-b10d-6a36561f085c" Jul 23 13:41:37.558354 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:41:37.558162 2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6kzs9" podUID="13853914-f73c-465d-a337-a8bbd8868adb" Jul 23 13:41:37.750673 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.750651 2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-225.ec2.internal" event="NodeReady" Jul 23 13:41:37.750837 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.750763 2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 13:41:37.798505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.798444 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6fb5c8cbbc-v957v"] Jul 23 13:41:37.809004 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.808970 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.811474 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.811450 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9kx5c\"" Jul 23 13:41:37.811474 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.811461 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 13:41:37.811686 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.811674 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 13:41:37.811760 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.811746 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 13:41:37.814449 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.814402 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wndd6"] Jul 23 13:41:37.821597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.821566 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 13:41:37.826598 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.826580 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pdx59"] Jul 23 13:41:37.826711 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.826696 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:37.829425 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.829405 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 13:41:37.829548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.829445 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-l4fll\"" Jul 23 13:41:37.829548 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.829532 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 13:41:37.836734 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.836710 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb5c8cbbc-v957v"] Jul 23 13:41:37.836824 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.836745 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wndd6"] Jul 23 13:41:37.836824 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.836785 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pdx59"] Jul 23 13:41:37.836962 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.836879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:37.839459 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.839442 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dmxg7\"" Jul 23 13:41:37.839670 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.839655 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 13:41:37.839769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.839753 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 13:41:37.839829 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.839800 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 13:41:37.842788 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.842765 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8ffrm"] Jul 23 13:41:37.853761 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.853741 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:37.856297 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.856273 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5fzgx\"" Jul 23 13:41:37.856413 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.856400 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 13:41:37.856475 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.856401 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 13:41:37.856475 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.856435 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 13:41:37.856475 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.856434 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 13:41:37.859818 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.859796 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8ffrm"] Jul 23 13:41:37.934490 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934467 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c6878de-2105-4646-9e71-5771b0fb9834-metrics-tls\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:37.934601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934503 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f571e772-75de-477b-8383-a46e1de71d1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:37.934601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934532 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f571e772-75de-477b-8383-a46e1de71d1d-data-volume\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:37.934601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/582be53d-6561-48f7-aebb-7e3be8c9a208-trusted-ca\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934575 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f571e772-75de-477b-8383-a46e1de71d1d-crio-socket\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:37.934601 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934591 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-registry-tls\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934605 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/582be53d-6561-48f7-aebb-7e3be8c9a208-installation-pull-secrets\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934622 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w8qc\" (UniqueName: \"kubernetes.io/projected/25122a4b-972d-4b3c-aca2-5d6bee9cb90b-kube-api-access-8w8qc\") pod \"ingress-canary-pdx59\" (UID: \"25122a4b-972d-4b3c-aca2-5d6bee9cb90b\") " pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:37.934768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934658 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f571e772-75de-477b-8383-a46e1de71d1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:37.934768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934708 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c6878de-2105-4646-9e71-5771b0fb9834-config-volume\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:37.934768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934738 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6878de-2105-4646-9e71-5771b0fb9834-tmp-dir\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:37.934768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934755 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/582be53d-6561-48f7-aebb-7e3be8c9a208-image-registry-private-configuration\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934779 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/582be53d-6561-48f7-aebb-7e3be8c9a208-ca-trust-extracted\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934798 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/582be53d-6561-48f7-aebb-7e3be8c9a208-registry-certificates\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934815 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggmdb\" (UniqueName: \"kubernetes.io/projected/f571e772-75de-477b-8383-a46e1de71d1d-kube-api-access-ggmdb\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934829 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-bound-sa-token\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934862 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbtv7\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-kube-api-access-qbtv7\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934904 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6974\" (UniqueName: \"kubernetes.io/projected/9c6878de-2105-4646-9e71-5771b0fb9834-kube-api-access-c6974\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:37.934964 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:37.934948 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25122a4b-972d-4b3c-aca2-5d6bee9cb90b-cert\") pod \"ingress-canary-pdx59\" (UID: \"25122a4b-972d-4b3c-aca2-5d6bee9cb90b\") " pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:38.035265 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035229 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c6878de-2105-4646-9e71-5771b0fb9834-config-volume\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.035371 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6878de-2105-4646-9e71-5771b0fb9834-tmp-dir\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.035371 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035298 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/582be53d-6561-48f7-aebb-7e3be8c9a208-image-registry-private-configuration\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.035371 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035322 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/582be53d-6561-48f7-aebb-7e3be8c9a208-ca-trust-extracted\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.035371 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/582be53d-6561-48f7-aebb-7e3be8c9a208-registry-certificates\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.035532 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035371 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggmdb\" (UniqueName: \"kubernetes.io/projected/f571e772-75de-477b-8383-a46e1de71d1d-kube-api-access-ggmdb\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.035532 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035395 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-bound-sa-token\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.035532 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035416 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbtv7\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-kube-api-access-qbtv7\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.035532 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035444 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6974\" (UniqueName: \"kubernetes.io/projected/9c6878de-2105-4646-9e71-5771b0fb9834-kube-api-access-c6974\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035531 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25122a4b-972d-4b3c-aca2-5d6bee9cb90b-cert\") pod \"ingress-canary-pdx59\" (UID: \"25122a4b-972d-4b3c-aca2-5d6bee9cb90b\") " pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035594 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c6878de-2105-4646-9e71-5771b0fb9834-metrics-tls\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035628 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f571e772-75de-477b-8383-a46e1de71d1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035660 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f571e772-75de-477b-8383-a46e1de71d1d-data-volume\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035687 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/582be53d-6561-48f7-aebb-7e3be8c9a208-trusted-ca\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035707 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6878de-2105-4646-9e71-5771b0fb9834-tmp-dir\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.035720 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035722 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f571e772-75de-477b-8383-a46e1de71d1d-crio-socket\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035736 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/582be53d-6561-48f7-aebb-7e3be8c9a208-ca-trust-extracted\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035744 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-registry-tls\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035768 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/582be53d-6561-48f7-aebb-7e3be8c9a208-installation-pull-secrets\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035833 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w8qc\" (UniqueName: \"kubernetes.io/projected/25122a4b-972d-4b3c-aca2-5d6bee9cb90b-kube-api-access-8w8qc\") pod \"ingress-canary-pdx59\" (UID: \"25122a4b-972d-4b3c-aca2-5d6bee9cb90b\") " pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.035875 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f571e772-75de-477b-8383-a46e1de71d1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.036002 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9c6878de-2105-4646-9e71-5771b0fb9834-config-volume\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.036137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.036066 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f571e772-75de-477b-8383-a46e1de71d1d-data-volume\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.036452 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.036216 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f571e772-75de-477b-8383-a46e1de71d1d-crio-socket\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.036513 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.036491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f571e772-75de-477b-8383-a46e1de71d1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.036995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.036941 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/582be53d-6561-48f7-aebb-7e3be8c9a208-registry-certificates\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.037570 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.037541 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/582be53d-6561-48f7-aebb-7e3be8c9a208-trusted-ca\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.040093 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.040068 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9c6878de-2105-4646-9e71-5771b0fb9834-metrics-tls\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.040172 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.040126 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f571e772-75de-477b-8383-a46e1de71d1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.040172 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.040151 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25122a4b-972d-4b3c-aca2-5d6bee9cb90b-cert\") pod \"ingress-canary-pdx59\" (UID: \"25122a4b-972d-4b3c-aca2-5d6bee9cb90b\") " pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:38.040297 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.040200 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/582be53d-6561-48f7-aebb-7e3be8c9a208-installation-pull-secrets\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.040297 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.040257 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-registry-tls\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.040509 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.040491 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/582be53d-6561-48f7-aebb-7e3be8c9a208-image-registry-private-configuration\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.044392 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.044339 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-bound-sa-token\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.045448 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.045426 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggmdb\" (UniqueName: \"kubernetes.io/projected/f571e772-75de-477b-8383-a46e1de71d1d-kube-api-access-ggmdb\") pod \"insights-runtime-extractor-8ffrm\" (UID: \"f571e772-75de-477b-8383-a46e1de71d1d\") " pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.045708 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.045691 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6974\" (UniqueName: \"kubernetes.io/projected/9c6878de-2105-4646-9e71-5771b0fb9834-kube-api-access-c6974\") pod \"dns-default-wndd6\" (UID: \"9c6878de-2105-4646-9e71-5771b0fb9834\") " pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.046041 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.046014 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbtv7\" (UniqueName: \"kubernetes.io/projected/582be53d-6561-48f7-aebb-7e3be8c9a208-kube-api-access-qbtv7\") pod \"image-registry-6fb5c8cbbc-v957v\" (UID: \"582be53d-6561-48f7-aebb-7e3be8c9a208\") " pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.046415 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.046388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w8qc\" (UniqueName: \"kubernetes.io/projected/25122a4b-972d-4b3c-aca2-5d6bee9cb90b-kube-api-access-8w8qc\") pod \"ingress-canary-pdx59\" (UID: \"25122a4b-972d-4b3c-aca2-5d6bee9cb90b\") " pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:38.122953 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.122923 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.136057 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.136032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:38.145088 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.145066 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pdx59" Jul 23 13:41:38.162410 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.162386 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8ffrm" Jul 23 13:41:38.336605 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.336560 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6fb5c8cbbc-v957v"] Jul 23 13:41:38.341959 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:38.341929 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod582be53d_6561_48f7_aebb_7e3be8c9a208.slice/crio-94b819a92dbd3f3f9d8a76538949799ca1ce5853ea746a47eb8554a7ca2b0137 WatchSource:0}: Error finding container 94b819a92dbd3f3f9d8a76538949799ca1ce5853ea746a47eb8554a7ca2b0137: Status 404 returned error can't find the container with id 94b819a92dbd3f3f9d8a76538949799ca1ce5853ea746a47eb8554a7ca2b0137 Jul 23 13:41:38.359874 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.359833 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pdx59"] Jul 23 13:41:38.363329 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.363307 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wndd6"] Jul 23 13:41:38.363960 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:38.363936 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25122a4b_972d_4b3c_aca2_5d6bee9cb90b.slice/crio-2d8468acc3dacadb062e9b23b6524e21941eece5dc0d7b61d2588a3716bffca3 WatchSource:0}: Error finding container 2d8468acc3dacadb062e9b23b6524e21941eece5dc0d7b61d2588a3716bffca3: Status 404 returned error can't find the container with id 2d8468acc3dacadb062e9b23b6524e21941eece5dc0d7b61d2588a3716bffca3 Jul 23 13:41:38.365906 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:38.365885 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c6878de_2105_4646_9e71_5771b0fb9834.slice/crio-92ce1d7f8257f88c62ed1323c52277eca53f903dcf4ef849a1c8ee16d5c0ad1b WatchSource:0}: Error finding container 92ce1d7f8257f88c62ed1323c52277eca53f903dcf4ef849a1c8ee16d5c0ad1b: Status 404 returned error can't find the container with id 92ce1d7f8257f88c62ed1323c52277eca53f903dcf4ef849a1c8ee16d5c0ad1b Jul 23 13:41:38.371366 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.371345 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8ffrm"] Jul 23 13:41:38.376817 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:38.376795 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf571e772_75de_477b_8383_a46e1de71d1d.slice/crio-ab2b1707939d132104dfca750882ad8c814367f7d2947b1ded5354ff4c666dd3 WatchSource:0}: Error finding container ab2b1707939d132104dfca750882ad8c814367f7d2947b1ded5354ff4c666dd3: Status 404 returned error can't find the container with id ab2b1707939d132104dfca750882ad8c814367f7d2947b1ded5354ff4c666dd3 Jul 23 13:41:38.911117 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.911036 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ffrm" event={"ID":"f571e772-75de-477b-8383-a46e1de71d1d","Type":"ContainerStarted","Data":"ab2b1707939d132104dfca750882ad8c814367f7d2947b1ded5354ff4c666dd3"} Jul 23 13:41:38.912446 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.912410 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pdx59" event={"ID":"25122a4b-972d-4b3c-aca2-5d6bee9cb90b","Type":"ContainerStarted","Data":"2d8468acc3dacadb062e9b23b6524e21941eece5dc0d7b61d2588a3716bffca3"} Jul 23 13:41:38.914815 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.914741 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" event={"ID":"582be53d-6561-48f7-aebb-7e3be8c9a208","Type":"ContainerStarted","Data":"48acee034c8e72786d1aa066e028ba9c85c296b5d7d9a4f8a11bb29df8754853"} Jul 23 13:41:38.914815 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.914786 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" event={"ID":"582be53d-6561-48f7-aebb-7e3be8c9a208","Type":"ContainerStarted","Data":"94b819a92dbd3f3f9d8a76538949799ca1ce5853ea746a47eb8554a7ca2b0137"} Jul 23 13:41:38.914815 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.914816 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:41:38.916043 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.916005 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wndd6" event={"ID":"9c6878de-2105-4646-9e71-5771b0fb9834","Type":"ContainerStarted","Data":"92ce1d7f8257f88c62ed1323c52277eca53f903dcf4ef849a1c8ee16d5c0ad1b"} Jul 23 13:41:38.938981 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:38.938931 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" podStartSLOduration=1.9389123750000001 podStartE2EDuration="1.938912375s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:38.937759356 +0000 UTC m=+50.959471236" watchObservedRunningTime="2026-07-23 13:41:38.938912375 +0000 UTC m=+50.960624250" Jul 23 13:41:39.557531 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.557497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:39.557699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.557536 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:39.557699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.557497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:39.562231 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.562188 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 13:41:39.562429 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.562409 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 13:41:39.562602 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.562589 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5nz2m\"" Jul 23 13:41:39.562820 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.562806 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 13:41:39.562998 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.562981 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pwdgr\"" Jul 23 13:41:39.563049 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:39.563012 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 13:41:45.936025 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.935983 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wndd6" event={"ID":"9c6878de-2105-4646-9e71-5771b0fb9834","Type":"ContainerStarted","Data":"2a39605d7c9bd6ba5a3dcee9b364b0bbe682af253bef5f3089b5d94284ef4f72"} Jul 23 13:41:45.936503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.936042 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wndd6" event={"ID":"9c6878de-2105-4646-9e71-5771b0fb9834","Type":"ContainerStarted","Data":"51963d441110795ff73971b093308b735a6f7202c4f645fb8cd5dfd8a57f5520"} Jul 23 13:41:45.936503 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.936107 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:45.937592 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.937568 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ffrm" event={"ID":"f571e772-75de-477b-8383-a46e1de71d1d","Type":"ContainerStarted","Data":"e2ab6bfcd24fcbe0dbae3f3fd0b528132e190600eee6c013174d73e6b149b6d6"} Jul 23 13:41:45.939144 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.939123 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pdx59" event={"ID":"25122a4b-972d-4b3c-aca2-5d6bee9cb90b","Type":"ContainerStarted","Data":"6f69a40e5bec5079c5788690ad788913d29ff7b7e2ca24eb31db66dbdcbd5c1d"} Jul 23 13:41:45.955046 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.955010 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wndd6" podStartSLOduration=2.30775473 podStartE2EDuration="8.954996995s" totalImagesPullingTime="6.647242265s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:45.954819495 +0000 UTC m=+57.976531374" watchObservedRunningTime="2026-07-23 13:41:45.954996995 +0000 UTC m=+57.976708873" Jul 23 13:41:45.972831 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:45.972774 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pdx59" podStartSLOduration=2.32249476 podStartE2EDuration="8.972747668s" totalImagesPullingTime="6.650252908s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:45.971636302 +0000 UTC m=+57.993348191" watchObservedRunningTime="2026-07-23 13:41:45.972747668 +0000 UTC m=+57.994459550" Jul 23 13:41:48.866240 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:48.866214 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7fwh6" Jul 23 13:41:48.949941 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:48.949915 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ffrm" event={"ID":"f571e772-75de-477b-8383-a46e1de71d1d","Type":"ContainerStarted","Data":"fc3d9aad51d84c09d689d7136cbb5f8e64cefbbb9c0f44cc0f7bac752c28c762"} Jul 23 13:41:50.287247 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.287215 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw"] Jul 23 13:41:50.289495 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.289474 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.291994 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.291970 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-v9d2m\"" Jul 23 13:41:50.292102 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.292010 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 13:41:50.292227 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.292213 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 13:41:50.292310 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.292295 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 13:41:50.292583 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.292561 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 13:41:50.292689 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.292677 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 13:41:50.299937 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.299917 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw"] Jul 23 13:41:50.333957 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.333925 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c59cf886-f0fb-4335-9254-46dcbf24ba51-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.334132 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.333993 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c59cf886-f0fb-4335-9254-46dcbf24ba51-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.334132 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.334036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cgd8\" (UniqueName: \"kubernetes.io/projected/c59cf886-f0fb-4335-9254-46dcbf24ba51-kube-api-access-2cgd8\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.334132 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.334081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59cf886-f0fb-4335-9254-46dcbf24ba51-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.434759 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.434718 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59cf886-f0fb-4335-9254-46dcbf24ba51-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.434937 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.434861 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c59cf886-f0fb-4335-9254-46dcbf24ba51-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.434937 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.434919 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c59cf886-f0fb-4335-9254-46dcbf24ba51-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.435056 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.434960 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cgd8\" (UniqueName: \"kubernetes.io/projected/c59cf886-f0fb-4335-9254-46dcbf24ba51-kube-api-access-2cgd8\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.435595 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.435567 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c59cf886-f0fb-4335-9254-46dcbf24ba51-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.437620 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.437587 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c59cf886-f0fb-4335-9254-46dcbf24ba51-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.437970 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.437925 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/c59cf886-f0fb-4335-9254-46dcbf24ba51-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.445135 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.445113 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cgd8\" (UniqueName: \"kubernetes.io/projected/c59cf886-f0fb-4335-9254-46dcbf24ba51-kube-api-access-2cgd8\") pod \"prometheus-operator-5945b5cfd-vvvkw\" (UID: \"c59cf886-f0fb-4335-9254-46dcbf24ba51\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.600821 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.600730 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" Jul 23 13:41:50.734881 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:50.734832 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw"] Jul 23 13:41:52.313572 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:52.313528 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc59cf886_f0fb_4335_9254_46dcbf24ba51.slice/crio-2de7732ab308bcd0a8ef0e522458e608c5832376cdd13798b74fb4e87d76c4e6 WatchSource:0}: Error finding container 2de7732ab308bcd0a8ef0e522458e608c5832376cdd13798b74fb4e87d76c4e6: Status 404 returned error can't find the container with id 2de7732ab308bcd0a8ef0e522458e608c5832376cdd13798b74fb4e87d76c4e6 Jul 23 13:41:52.968065 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:52.968029 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8ffrm" event={"ID":"f571e772-75de-477b-8383-a46e1de71d1d","Type":"ContainerStarted","Data":"a2d84d10744da22027cfaba221c6191af6d209344ff021b4fbe92ba51c08d08e"} Jul 23 13:41:52.969311 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:52.969280 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" event={"ID":"c59cf886-f0fb-4335-9254-46dcbf24ba51","Type":"ContainerStarted","Data":"2de7732ab308bcd0a8ef0e522458e608c5832376cdd13798b74fb4e87d76c4e6"} Jul 23 13:41:52.988886 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:52.988603 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8ffrm" podStartSLOduration=2.195076647 podStartE2EDuration="15.988584629s" totalImagesPullingTime="13.793507982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:41:37 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:52.987885116 +0000 UTC m=+65.009596996" watchObservedRunningTime="2026-07-23 13:41:52.988584629 +0000 UTC m=+65.010296509" Jul 23 13:41:53.974075 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:53.974026 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" event={"ID":"c59cf886-f0fb-4335-9254-46dcbf24ba51","Type":"ContainerStarted","Data":"ec0ce77965e6a32b448ea207dff3fd39e8024931273ef713468b4d2e239c96a2"} Jul 23 13:41:53.974455 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:53.974120 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" event={"ID":"c59cf886-f0fb-4335-9254-46dcbf24ba51","Type":"ContainerStarted","Data":"9c0baf2e32c00415ec506963561ad3eb9e9c22585d368135b2afa3ccceddc7f0"} Jul 23 13:41:53.996896 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:53.996827 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-vvvkw" podStartSLOduration=2.732645171 podStartE2EDuration="3.996808595s" totalImagesPullingTime="1.264163424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:53.996339886 +0000 UTC m=+66.018051768" watchObservedRunningTime="2026-07-23 13:41:53.996808595 +0000 UTC m=+66.018520477" Jul 23 13:41:54.166626 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.166583 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:54.169782 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.169760 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 13:41:54.179074 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.179045 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2618d6b8-b672-4af6-bc7d-5e13639fc6a3-original-pull-secret\") pod \"global-pull-secret-syncer-rv5cp\" (UID: \"2618d6b8-b672-4af6-bc7d-5e13639fc6a3\") " pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:54.267372 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.267287 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:54.267372 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.267326 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:54.270138 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.270106 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 13:41:54.270264 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.270219 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 13:41:54.279731 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.279710 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13853914-f73c-465d-a337-a8bbd8868adb-metrics-certs\") pod \"network-metrics-daemon-6kzs9\" (UID: \"13853914-f73c-465d-a337-a8bbd8868adb\") " pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:54.280087 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.280074 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 13:41:54.288421 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.288405 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rv5cp" Jul 23 13:41:54.290639 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.290620 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnbpv\" (UniqueName: \"kubernetes.io/projected/e08e055d-d7bb-4173-b10d-6a36561f085c-kube-api-access-nnbpv\") pod \"network-check-target-rc6nl\" (UID: \"e08e055d-d7bb-4173-b10d-6a36561f085c\") " pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:54.406439 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.406412 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rv5cp"] Jul 23 13:41:54.409403 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:54.409369 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2618d6b8_b672_4af6_bc7d_5e13639fc6a3.slice/crio-e47bcca71ca4fbd4c6379722006f131b7a83ee950d9052ac37b781215072b9de WatchSource:0}: Error finding container e47bcca71ca4fbd4c6379722006f131b7a83ee950d9052ac37b781215072b9de: Status 404 returned error can't find the container with id e47bcca71ca4fbd4c6379722006f131b7a83ee950d9052ac37b781215072b9de Jul 23 13:41:54.575831 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.575765 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-pwdgr\"" Jul 23 13:41:54.583128 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.583113 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5nz2m\"" Jul 23 13:41:54.583195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.583170 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6kzs9" Jul 23 13:41:54.591657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.591640 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:54.714752 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.714727 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6kzs9"] Jul 23 13:41:54.717927 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:54.717899 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13853914_f73c_465d_a337_a8bbd8868adb.slice/crio-152cb4b0751835d09cf921af0fd19c3fb13bfa291b04922f8cdc4ef731dd4f5a WatchSource:0}: Error finding container 152cb4b0751835d09cf921af0fd19c3fb13bfa291b04922f8cdc4ef731dd4f5a: Status 404 returned error can't find the container with id 152cb4b0751835d09cf921af0fd19c3fb13bfa291b04922f8cdc4ef731dd4f5a Jul 23 13:41:54.729135 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.729114 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rc6nl"] Jul 23 13:41:54.731423 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:54.731396 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08e055d_d7bb_4173_b10d_6a36561f085c.slice/crio-473948e8494e7d604c4cb335849db88e4ecbb5b55e44f581392d736f9a07df79 WatchSource:0}: Error finding container 473948e8494e7d604c4cb335849db88e4ecbb5b55e44f581392d736f9a07df79: Status 404 returned error can't find the container with id 473948e8494e7d604c4cb335849db88e4ecbb5b55e44f581392d736f9a07df79 Jul 23 13:41:54.978297 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.978261 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6kzs9" event={"ID":"13853914-f73c-465d-a337-a8bbd8868adb","Type":"ContainerStarted","Data":"152cb4b0751835d09cf921af0fd19c3fb13bfa291b04922f8cdc4ef731dd4f5a"} Jul 23 13:41:54.979336 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.979312 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rv5cp" event={"ID":"2618d6b8-b672-4af6-bc7d-5e13639fc6a3","Type":"ContainerStarted","Data":"e47bcca71ca4fbd4c6379722006f131b7a83ee950d9052ac37b781215072b9de"} Jul 23 13:41:54.980387 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:54.980363 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rc6nl" event={"ID":"e08e055d-d7bb-4173-b10d-6a36561f085c","Type":"ContainerStarted","Data":"473948e8494e7d604c4cb335849db88e4ecbb5b55e44f581392d736f9a07df79"} Jul 23 13:41:55.684162 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.684128 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wwjhj"] Jul 23 13:41:55.686831 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.686801 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.689947 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.689536 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j9cvs\"" Jul 23 13:41:55.689947 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.689609 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 13:41:55.689947 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.689759 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 13:41:55.689947 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.689823 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 13:41:55.779748 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.779713 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-tls\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.779949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.779754 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw4gr\" (UniqueName: \"kubernetes.io/projected/800e28ee-361c-4a25-9120-9d33513a56d2-kube-api-access-cw4gr\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.779949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.779782 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-textfile\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.779949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.779859 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-wtmp\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.779949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.779934 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.780137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.779973 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/800e28ee-361c-4a25-9120-9d33513a56d2-metrics-client-ca\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.780137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.780013 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.780137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.780036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-root\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.780137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.780114 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-sys\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881266 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881432 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881276 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/800e28ee-361c-4a25-9120-9d33513a56d2-metrics-client-ca\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881432 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881432 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881332 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-root\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881432 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881370 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-sys\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881432 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881408 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-tls\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881432 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw4gr\" (UniqueName: \"kubernetes.io/projected/800e28ee-361c-4a25-9120-9d33513a56d2-kube-api-access-cw4gr\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881453 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-textfile\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881469 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-wtmp\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.881688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.881629 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-wtmp\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.882749 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.882203 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-textfile\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.882749 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.882527 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-sys\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.882749 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.882588 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/800e28ee-361c-4a25-9120-9d33513a56d2-root\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.882749 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.882706 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.883425 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.882926 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/800e28ee-361c-4a25-9120-9d33513a56d2-metrics-client-ca\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.884477 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.884451 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.884710 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.884685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/800e28ee-361c-4a25-9120-9d33513a56d2-node-exporter-tls\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.892426 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.892376 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw4gr\" (UniqueName: \"kubernetes.io/projected/800e28ee-361c-4a25-9120-9d33513a56d2-kube-api-access-cw4gr\") pod \"node-exporter-wwjhj\" (UID: \"800e28ee-361c-4a25-9120-9d33513a56d2\") " pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:55.945352 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.945327 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wndd6" Jul 23 13:41:55.990483 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.990449 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6kzs9" event={"ID":"13853914-f73c-465d-a337-a8bbd8868adb","Type":"ContainerStarted","Data":"be316cabbebaded98e7c37d5bc5730b251a2b449d3adb37cf7ddbb207eb9dc92"} Jul 23 13:41:55.999059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:55.998805 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wwjhj" Jul 23 13:41:56.012316 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:56.012289 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800e28ee_361c_4a25_9120_9d33513a56d2.slice/crio-0a66f9b3ff445c2ece5ca92d8b0e756302a3395f3280de93967f33e1578d5e00 WatchSource:0}: Error finding container 0a66f9b3ff445c2ece5ca92d8b0e756302a3395f3280de93967f33e1578d5e00: Status 404 returned error can't find the container with id 0a66f9b3ff445c2ece5ca92d8b0e756302a3395f3280de93967f33e1578d5e00 Jul 23 13:41:57.000963 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:57.000926 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6kzs9" event={"ID":"13853914-f73c-465d-a337-a8bbd8868adb","Type":"ContainerStarted","Data":"dc0f1fe31392b050f46770df1f1f7dbc282c51e24b2e79e8a2deb5ed29593474"} Jul 23 13:41:57.002470 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:57.002439 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wwjhj" event={"ID":"800e28ee-361c-4a25-9120-9d33513a56d2","Type":"ContainerStarted","Data":"0a66f9b3ff445c2ece5ca92d8b0e756302a3395f3280de93967f33e1578d5e00"} Jul 23 13:41:57.019038 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:57.018995 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6kzs9" podStartSLOduration=66.855139079 podStartE2EDuration="1m8.018983414s" totalImagesPullingTime="1.163844335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:57.018726264 +0000 UTC m=+69.040438144" watchObservedRunningTime="2026-07-23 13:41:57.018983414 +0000 UTC m=+69.040695293" Jul 23 13:41:58.127988 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.127823 2582 patch_prober.go:28] interesting pod/image-registry-6fb5c8cbbc-v957v container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 23 13:41:58.127988 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.127904 2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" podUID="582be53d-6561-48f7-aebb-7e3be8c9a208" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:41:58.665752 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.665677 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn"] Jul 23 13:41:58.674935 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.674879 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.677763 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.677745 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 13:41:58.677926 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.677801 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 13:41:58.678113 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.678087 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 13:41:58.678113 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.678101 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h4plf\"" Jul 23 13:41:58.678709 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.678689 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-70oc3q3dq1f3h\"" Jul 23 13:41:58.678798 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.678727 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 13:41:58.678798 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.678729 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 13:41:58.683896 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.683882 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn"] Jul 23 13:41:58.700483 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700454 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700618 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700500 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700618 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700618 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700550 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-grpc-tls\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700618 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700575 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96abc0a4-e39a-461c-9177-d0b845c4aed4-metrics-client-ca\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700829 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700656 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-tls\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700829 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700689 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.700829 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.700768 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zx9n\" (UniqueName: \"kubernetes.io/projected/96abc0a4-e39a-461c-9177-d0b845c4aed4-kube-api-access-6zx9n\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801489 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zx9n\" (UniqueName: \"kubernetes.io/projected/96abc0a4-e39a-461c-9177-d0b845c4aed4-kube-api-access-6zx9n\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801646 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801500 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801646 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801529 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801646 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801556 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801646 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-grpc-tls\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801646 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96abc0a4-e39a-461c-9177-d0b845c4aed4-metrics-client-ca\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.801646 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801636 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-tls\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.802030 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.801667 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.802770 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.802740 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/96abc0a4-e39a-461c-9177-d0b845c4aed4-metrics-client-ca\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.804741 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.804691 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.804741 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.804691 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.804921 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.804878 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-grpc-tls\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.804921 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.804886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-tls\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.805368 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.805345 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.806211 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.806187 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/96abc0a4-e39a-461c-9177-d0b845c4aed4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.810633 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.810615 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zx9n\" (UniqueName: \"kubernetes.io/projected/96abc0a4-e39a-461c-9177-d0b845c4aed4-kube-api-access-6zx9n\") pod \"thanos-querier-747ddb4dbd-zt5qn\" (UID: \"96abc0a4-e39a-461c-9177-d0b845c4aed4\") " pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:58.987352 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:58.987273 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:41:59.011464 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.011424 2582 generic.go:356] "Generic (PLEG): container finished" podID="800e28ee-361c-4a25-9120-9d33513a56d2" containerID="18feaa1ecd632eef664a31f8b1c14d1b6203c3086ccd61382427d4512125747c" exitCode=0 Jul 23 13:41:59.011615 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.011517 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wwjhj" event={"ID":"800e28ee-361c-4a25-9120-9d33513a56d2","Type":"ContainerDied","Data":"18feaa1ecd632eef664a31f8b1c14d1b6203c3086ccd61382427d4512125747c"} Jul 23 13:41:59.013272 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.013248 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rc6nl" event={"ID":"e08e055d-d7bb-4173-b10d-6a36561f085c","Type":"ContainerStarted","Data":"6df9a5e05df28bf3db8d46dddc747be5495efff7f651e2eccd398747b7e9d7e7"} Jul 23 13:41:59.013417 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.013385 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:41:59.047976 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.047933 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rc6nl" podStartSLOduration=66.440227069 podStartE2EDuration="1m10.047917417s" totalImagesPullingTime="3.607690348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:41:59.047394645 +0000 UTC m=+71.069106524" watchObservedRunningTime="2026-07-23 13:41:59.047917417 +0000 UTC m=+71.069629296" Jul 23 13:41:59.169836 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.169696 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn"] Jul 23 13:41:59.172168 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:41:59.172140 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96abc0a4_e39a_461c_9177_d0b845c4aed4.slice/crio-157f014ef5cd974bda095453a799ea34253249e49b5737d855ea9eb2bf3f9d8e WatchSource:0}: Error finding container 157f014ef5cd974bda095453a799ea34253249e49b5737d855ea9eb2bf3f9d8e: Status 404 returned error can't find the container with id 157f014ef5cd974bda095453a799ea34253249e49b5737d855ea9eb2bf3f9d8e Jul 23 13:41:59.924235 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:41:59.924207 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6fb5c8cbbc-v957v" Jul 23 13:42:00.017746 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.017716 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64d8476f76-z2ztf"] Jul 23 13:42:00.020516 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.020081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.021929 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.021782 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wwjhj" event={"ID":"800e28ee-361c-4a25-9120-9d33513a56d2","Type":"ContainerStarted","Data":"b6c645119d9b7a1b9dd57b65a1ce1bc69597466d9080c572b72dd3973c4bef1b"} Jul 23 13:42:00.021929 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.021830 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wwjhj" event={"ID":"800e28ee-361c-4a25-9120-9d33513a56d2","Type":"ContainerStarted","Data":"60e646d965a47891e2c59a712eaad66df68602fb55597ed1c8104aa1cf6f2d7a"} Jul 23 13:42:00.024078 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.023937 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"157f014ef5cd974bda095453a799ea34253249e49b5737d855ea9eb2bf3f9d8e"} Jul 23 13:42:00.027433 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.026759 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 23 13:42:00.027433 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.027064 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 23 13:42:00.027433 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.027291 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-dbrvr\"" Jul 23 13:42:00.027659 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.027434 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 13:42:00.028067 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.027879 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 23 13:42:00.032403 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.032384 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-db1igu7rudjif\"" Jul 23 13:42:00.043131 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.043107 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64d8476f76-z2ztf"] Jul 23 13:42:00.082288 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.082231 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wwjhj" podStartSLOduration=2.64882944 podStartE2EDuration="5.082211522s" totalImagesPullingTime="2.329125724s" totalInitContainerRuntime="104.256358ms" isStatefulPod=true podCreationTimestamp="2026-07-23 13:41:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:42:00.081280217 +0000 UTC m=+72.102992121" watchObservedRunningTime="2026-07-23 13:42:00.082211522 +0000 UTC m=+72.103923403" Jul 23 13:42:00.112390 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112356 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-audit-log\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.112390 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112393 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-metrics-server-audit-profiles\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.112623 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112481 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-client-ca-bundle\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.112623 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112528 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-secret-metrics-server-tls\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.112623 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112558 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k6pn\" (UniqueName: \"kubernetes.io/projected/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-kube-api-access-8k6pn\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.112623 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112590 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.112868 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.112724 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-secret-metrics-server-client-certs\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.213822 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213688 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.213822 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213754 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-secret-metrics-server-client-certs\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.213822 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213795 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-audit-log\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.214409 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213825 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-metrics-server-audit-profiles\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.214409 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-client-ca-bundle\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.214409 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-secret-metrics-server-tls\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.214409 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.213947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k6pn\" (UniqueName: \"kubernetes.io/projected/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-kube-api-access-8k6pn\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.214615 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.214564 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.215165 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.215118 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-audit-log\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.215475 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.215433 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-metrics-server-audit-profiles\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.217497 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.217471 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-client-ca-bundle\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.218413 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.218370 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-secret-metrics-server-client-certs\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.219597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.219575 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-secret-metrics-server-tls\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.225197 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.225177 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k6pn\" (UniqueName: \"kubernetes.io/projected/389aaeb0-01ed-4b02-a94e-6c3b8992cc00-kube-api-access-8k6pn\") pod \"metrics-server-64d8476f76-z2ztf\" (UID: \"389aaeb0-01ed-4b02-a94e-6c3b8992cc00\") " pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.333324 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.333171 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:00.489634 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:00.489549 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64d8476f76-z2ztf"] Jul 23 13:42:00.493498 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:42:00.493470 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod389aaeb0_01ed_4b02_a94e_6c3b8992cc00.slice/crio-046986e30a34b36855ef5955e8cbac74c2e2fc0534ff68f60756e392253d041d WatchSource:0}: Error finding container 046986e30a34b36855ef5955e8cbac74c2e2fc0534ff68f60756e392253d041d: Status 404 returned error can't find the container with id 046986e30a34b36855ef5955e8cbac74c2e2fc0534ff68f60756e392253d041d Jul 23 13:42:01.028429 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.028387 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" event={"ID":"389aaeb0-01ed-4b02-a94e-6c3b8992cc00","Type":"ContainerStarted","Data":"046986e30a34b36855ef5955e8cbac74c2e2fc0534ff68f60756e392253d041d"} Jul 23 13:42:01.917102 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.917057 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:42:01.920316 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.920286 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:01.925992 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.925956 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 13:42:01.926212 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.926186 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 13:42:01.926687 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.926485 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 13:42:01.926687 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.926522 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 13:42:01.927174 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.927156 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 13:42:01.927403 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.927386 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 13:42:01.928220 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.927945 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sbv2k\"" Jul 23 13:42:01.928220 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928141 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 13:42:01.928380 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928330 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2bvekbfu2372u\"" Jul 23 13:42:01.928553 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928524 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 13:42:01.928927 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928679 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 13:42:01.928927 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928739 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 13:42:01.928927 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928805 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 13:42:01.928927 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.928881 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 13:42:01.934340 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.934294 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 13:42:01.936291 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.936272 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 13:42:01.955996 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:01.955958 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:42:02.027476 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027421 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027476 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027468 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027525 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27xxz\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-kube-api-access-27xxz\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027570 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027600 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027619 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027696 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.027737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027729 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027759 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027786 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-web-config\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027836 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027906 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027955 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.027988 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config-out\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.028008 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.028029 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028460 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.028072 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.028460 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.028094 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129500 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config-out\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129686 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129510 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129686 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129542 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129686 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129634 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129686 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129720 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129750 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129775 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27xxz\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-kube-api-access-27xxz\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129807 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129831 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129877 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129905 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.129952 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.129950 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.135869 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.135917 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.135944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.135958 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.135980 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-web-config\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.136032 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.136070 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.136095 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136302 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.136115 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136860 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.136675 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136860 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.135802 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.136974 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.136960 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.137208 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.132589 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.137892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.137768 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.137892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.137789 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.137892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.132891 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.137892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.134046 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config-out\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.137892 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.132155 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.138303 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.138278 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.139368 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.139345 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.139724 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.139676 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.139904 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.139886 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.140114 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.140094 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.140204 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.140115 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-web-config\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.140597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.140580 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.142372 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.142347 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.145553 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.145510 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27xxz\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-kube-api-access-27xxz\") pod \"prometheus-k8s-0\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.236963 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.236860 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:02.957342 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:02.957204 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:42:02.964383 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:42:02.964350 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf185aa_ef2b_4e18_a83a_9601c3f1cc20.slice/crio-45e975ab612f6aeb18b8f138b021cd4ff32ab086a5c064570968764d05da69c3 WatchSource:0}: Error finding container 45e975ab612f6aeb18b8f138b021cd4ff32ab086a5c064570968764d05da69c3: Status 404 returned error can't find the container with id 45e975ab612f6aeb18b8f138b021cd4ff32ab086a5c064570968764d05da69c3 Jul 23 13:42:03.039735 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:03.039669 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rv5cp" event={"ID":"2618d6b8-b672-4af6-bc7d-5e13639fc6a3","Type":"ContainerStarted","Data":"d536a9de90a9f96dd9deff8683a0f1f779ecd5ce4424ffad67f2c4b8888db39f"} Jul 23 13:42:03.043620 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:03.043481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"277a0a45ab0ede5a97ad447934a7eb9228708a4baa2a11fd9a62be1c50b5c5e1"} Jul 23 13:42:03.043620 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:03.043558 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"39e246c9797686ba3ee9d8053ed538a17dd2f3295a119a22c1f1c5bbd4f03489"} Jul 23 13:42:03.047130 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:03.047044 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"45e975ab612f6aeb18b8f138b021cd4ff32ab086a5c064570968764d05da69c3"} Jul 23 13:42:03.057874 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:03.057612 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rv5cp" podStartSLOduration=65.678168128 podStartE2EDuration="1m14.057592502s" totalImagesPullingTime="8.379424374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 13:40:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:42:03.056692216 +0000 UTC m=+75.078404095" watchObservedRunningTime="2026-07-23 13:42:03.057592502 +0000 UTC m=+75.079304375" Jul 23 13:42:04.059733 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:04.059694 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"99a6dc918dd8d429496f2265a80827cf87c72d88b7e871b87a3b6a3f752bb682"} Jul 23 13:42:05.064600 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.064563 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" event={"ID":"389aaeb0-01ed-4b02-a94e-6c3b8992cc00","Type":"ContainerStarted","Data":"ab1eda6a51207d10281a50dab3339898cae69f2a60475fb0893cf8a7ed9d0f8c"} Jul 23 13:42:05.067493 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.067460 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"370b032555d664c89a971f10c71713c1418a17a5e86bdff3c09fda5f5374a0ee"} Jul 23 13:42:05.067615 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.067523 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"e20ee6d784e11f0b04ac1c4115f910be6fb7929e57d2d613a63de8cb5c508e1e"} Jul 23 13:42:05.067615 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.067551 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" event={"ID":"96abc0a4-e39a-461c-9177-d0b845c4aed4","Type":"ContainerStarted","Data":"4f9d0fc1a95dfd7cfa56d0d5e0f3b06618aeb557e90cb4f26571116e6f84002f"} Jul 23 13:42:05.067615 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.067592 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:42:05.073289 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.073265 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7" exitCode=0 Jul 23 13:42:05.073404 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.073308 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7"} Jul 23 13:42:05.086412 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.086372 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" podStartSLOduration=2.678416832 podStartE2EDuration="6.086359547s" totalImagesPullingTime="3.407942715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:42:05.084949674 +0000 UTC m=+77.106661584" watchObservedRunningTime="2026-07-23 13:42:05.086359547 +0000 UTC m=+77.108071426" Jul 23 13:42:05.147669 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:05.147610 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" podStartSLOduration=2.126289958 podStartE2EDuration="7.147585096s" totalImagesPullingTime="5.021295138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:41:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:42:05.112205886 +0000 UTC m=+77.133917765" watchObservedRunningTime="2026-07-23 13:42:05.147585096 +0000 UTC m=+77.169296977" Jul 23 13:42:10.098409 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:10.098378 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8"} Jul 23 13:42:10.098784 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:10.098452 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea"} Jul 23 13:42:11.082111 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:11.082083 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-747ddb4dbd-zt5qn" Jul 23 13:42:11.105196 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:11.105161 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833"} Jul 23 13:42:11.105558 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:11.105221 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460"} Jul 23 13:42:11.105558 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:11.105247 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d"} Jul 23 13:42:11.105558 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:11.105280 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerStarted","Data":"ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1"} Jul 23 13:42:11.197446 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:11.197393 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.045827581 podStartE2EDuration="10.197376274s" totalImagesPullingTime="6.130424564s" totalInitContainerRuntime="21.124129ms" isStatefulPod=false podCreationTimestamp="2026-07-23 13:42:01 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:42:11.196264293 +0000 UTC m=+83.217976191" watchObservedRunningTime="2026-07-23 13:42:11.197376274 +0000 UTC m=+83.219088152" Jul 23 13:42:12.237317 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:12.237285 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:42:20.333804 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:20.333769 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:20.334182 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:20.333873 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:30.027197 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:30.027166 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rc6nl" Jul 23 13:42:40.339070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:40.339041 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:42:40.342823 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:42:40.342801 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64d8476f76-z2ztf" Jul 23 13:43:02.237625 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:02.237592 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:02.258733 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:02.258706 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:02.308197 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:02.308167 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:20.307949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.307896 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:43:20.308544 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.308365 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-thanos" containerID="cri-o://3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833" gracePeriod=600 Jul 23 13:43:20.308717 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.308668 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="prometheus" containerID="cri-o://03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea" gracePeriod=600 Jul 23 13:43:20.308946 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.308725 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-web" containerID="cri-o://4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d" gracePeriod=600 Jul 23 13:43:20.309067 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.308754 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy" containerID="cri-o://e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460" gracePeriod=600 Jul 23 13:43:20.309129 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.308773 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="thanos-sidecar" containerID="cri-o://ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1" gracePeriod=600 Jul 23 13:43:20.309181 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:20.308790 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="config-reloader" containerID="cri-o://90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8" gracePeriod=600 Jul 23 13:43:21.364061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364029 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833" exitCode=0 Jul 23 13:43:21.364061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364054 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460" exitCode=0 Jul 23 13:43:21.364061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364061 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1" exitCode=0 Jul 23 13:43:21.364061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364067 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8" exitCode=0 Jul 23 13:43:21.364061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364071 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea" exitCode=0 Jul 23 13:43:21.364597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364101 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833"} Jul 23 13:43:21.364597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364164 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460"} Jul 23 13:43:21.364597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364190 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1"} Jul 23 13:43:21.364597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364212 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8"} Jul 23 13:43:21.364597 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.364236 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea"} Jul 23 13:43:21.540186 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.540164 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:21.693633 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693549 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-grpc-tls\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693633 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693579 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693633 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693600 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-metrics-client-certs\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693633 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693633 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-2\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693648 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693811 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-1\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693869 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config-out\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.693949 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693918 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-0\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693966 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-serving-certs-ca-bundle\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.693996 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27xxz\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-kube-api-access-27xxz\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694030 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-web-config\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694064 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-thanos-prometheus-http-client-file\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694095 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-kube-rbac-proxy\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694095 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694117 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-db\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694150 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-tls-assets\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694534 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694178 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-metrics-client-ca\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694534 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694205 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-tls\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694534 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694232 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-trusted-ca-bundle\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694534 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694258 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-kubelet-serving-ca-bundle\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694534 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694284 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\" (UID: \"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20\") " Jul 23 13:43:21.694534 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694399 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.694834 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694571 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.694834 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694592 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.694988 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.694955 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.696453 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.696121 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.696453 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.696178 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.697092 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.696794 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.697961 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.697359 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 13:43:21.697961 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.697622 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config" (OuterVolumeSpecName: "config") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.698288 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.698249 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-kube-api-access-27xxz" (OuterVolumeSpecName: "kube-api-access-27xxz") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "kube-api-access-27xxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:43:21.698288 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.698266 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:21.698288 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.698280 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config-out" (OuterVolumeSpecName: "config-out") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:43:21.698567 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.698542 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.698973 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.698948 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.699291 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.699250 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.699392 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.699337 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.699485 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.699457 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.699551 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.699501 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:43:21.699611 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.699577 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.700059 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.700038 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.708941 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.708856 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-web-config" (OuterVolumeSpecName: "web-config") pod "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" (UID: "8cf185aa-ef2b-4e18-a83a-9601c3f1cc20"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 13:43:21.795443 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795413 2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-grpc-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795443 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795438 2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795443 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795448 2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-metrics-client-certs\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795460 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795469 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795478 2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-config-out\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795500 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795510 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27xxz\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-kube-api-access-27xxz\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795519 2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-web-config\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795527 2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795537 2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-kube-rbac-proxy\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795545 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-k8s-db\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795553 2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-tls-assets\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795562 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-metrics-client-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795569 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795578 2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795587 2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:21.795642 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:21.795596 2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:43:22.370816 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.370778 2582 generic.go:356] "Generic (PLEG): container finished" podID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerID="4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d" exitCode=0 Jul 23 13:43:22.371195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.370837 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d"} Jul 23 13:43:22.371195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.370884 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.371195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.370929 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8cf185aa-ef2b-4e18-a83a-9601c3f1cc20","Type":"ContainerDied","Data":"45e975ab612f6aeb18b8f138b021cd4ff32ab086a5c064570968764d05da69c3"} Jul 23 13:43:22.371195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.370979 2582 scope.go:117] "RemoveContainer" containerID="3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833" Jul 23 13:43:22.378610 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.378594 2582 scope.go:117] "RemoveContainer" containerID="e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460" Jul 23 13:43:22.385374 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.385357 2582 scope.go:117] "RemoveContainer" containerID="4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d" Jul 23 13:43:22.391589 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.391571 2582 scope.go:117] "RemoveContainer" containerID="ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1" Jul 23 13:43:22.395440 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.395419 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:43:22.398776 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.398759 2582 scope.go:117] "RemoveContainer" containerID="90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8" Jul 23 13:43:22.400279 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.400259 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:43:22.405055 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.405039 2582 scope.go:117] "RemoveContainer" containerID="03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea" Jul 23 13:43:22.411550 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.411533 2582 scope.go:117] "RemoveContainer" containerID="b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7" Jul 23 13:43:22.417530 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.417513 2582 scope.go:117] "RemoveContainer" containerID="3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833" Jul 23 13:43:22.417767 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.417749 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833\": container with ID starting with 3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833 not found: ID does not exist" containerID="3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833" Jul 23 13:43:22.417814 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.417777 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833"} err="failed to get container status \"3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833\": rpc error: code = NotFound desc = could not find container \"3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833\": container with ID starting with 3daa83303ab2d764356007f320470b0c159902004326c6182b16f0b65ba7e833 not found: ID does not exist" Jul 23 13:43:22.417879 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.417816 2582 scope.go:117] "RemoveContainer" containerID="e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460" Jul 23 13:43:22.418133 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.418114 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460\": container with ID starting with e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460 not found: ID does not exist" containerID="e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460" Jul 23 13:43:22.418195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418142 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460"} err="failed to get container status \"e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460\": rpc error: code = NotFound desc = could not find container \"e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460\": container with ID starting with e6a573bf75c92c8199326db6c5c5f07bf32a2890888f1e1c9906165149dc5460 not found: ID does not exist" Jul 23 13:43:22.418195 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418164 2582 scope.go:117] "RemoveContainer" containerID="4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d" Jul 23 13:43:22.418414 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.418398 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d\": container with ID starting with 4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d not found: ID does not exist" containerID="4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d" Jul 23 13:43:22.418465 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418420 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d"} err="failed to get container status \"4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d\": rpc error: code = NotFound desc = could not find container \"4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d\": container with ID starting with 4a46f68a5d6dedc1a7d7fdc5435dac3fc4b4d37389092d616f0fa30df65f7b3d not found: ID does not exist" Jul 23 13:43:22.418465 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418435 2582 scope.go:117] "RemoveContainer" containerID="ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1" Jul 23 13:43:22.418661 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.418633 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1\": container with ID starting with ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1 not found: ID does not exist" containerID="ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1" Jul 23 13:43:22.418724 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418669 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1"} err="failed to get container status \"ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1\": rpc error: code = NotFound desc = could not find container \"ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1\": container with ID starting with ab2833e3d51f1133425adbf7e18e258fd2738bf512bb9fc95cfced7d2579eef1 not found: ID does not exist" Jul 23 13:43:22.418724 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418688 2582 scope.go:117] "RemoveContainer" containerID="90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8" Jul 23 13:43:22.418943 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.418924 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8\": container with ID starting with 90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8 not found: ID does not exist" containerID="90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8" Jul 23 13:43:22.418987 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418948 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8"} err="failed to get container status \"90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8\": rpc error: code = NotFound desc = could not find container \"90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8\": container with ID starting with 90288520bc2b8a5bfa0a6cecdb2c6a19fde47983066e1af159cfcf29d13b58e8 not found: ID does not exist" Jul 23 13:43:22.418987 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.418963 2582 scope.go:117] "RemoveContainer" containerID="03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea" Jul 23 13:43:22.419169 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.419153 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea\": container with ID starting with 03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea not found: ID does not exist" containerID="03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea" Jul 23 13:43:22.419210 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.419173 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea"} err="failed to get container status \"03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea\": rpc error: code = NotFound desc = could not find container \"03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea\": container with ID starting with 03e0e828fd6388b1fd015a9010e06f4b1bd447a51189536b553a07a9453a4bea not found: ID does not exist" Jul 23 13:43:22.419210 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.419186 2582 scope.go:117] "RemoveContainer" containerID="b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7" Jul 23 13:43:22.419414 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:43:22.419398 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7\": container with ID starting with b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7 not found: ID does not exist" containerID="b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7" Jul 23 13:43:22.419470 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.419430 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7"} err="failed to get container status \"b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7\": rpc error: code = NotFound desc = could not find container \"b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7\": container with ID starting with b478c067684a5c772edfd22b4506f031ef337a96cb9d71ec5ea976c09d1e9ea7 not found: ID does not exist" Jul 23 13:43:22.428307 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428289 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:43:22.428520 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428509 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-web" Jul 23 13:43:22.428556 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428522 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-web" Jul 23 13:43:22.428556 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428533 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="config-reloader" Jul 23 13:43:22.428556 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428538 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="config-reloader" Jul 23 13:43:22.428556 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428547 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="init-config-reloader" Jul 23 13:43:22.428556 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428552 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="init-config-reloader" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428560 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="thanos-sidecar" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428566 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="thanos-sidecar" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428573 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-thanos" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428578 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-thanos" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428587 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428592 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428598 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="prometheus" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428603 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="prometheus" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428640 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="thanos-sidecar" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428648 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-web" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428654 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy-thanos" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428659 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="prometheus" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428664 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="kube-rbac-proxy" Jul 23 13:43:22.428696 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.428669 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" containerName="config-reloader" Jul 23 13:43:22.433925 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.433904 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.436586 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.436565 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 23 13:43:22.436688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.436582 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sbv2k\"" Jul 23 13:43:22.436688 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.436582 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 23 13:43:22.436900 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.436886 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 23 13:43:22.437189 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437164 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 23 13:43:22.437189 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437224 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 23 13:43:22.437478 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437383 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2bvekbfu2372u\"" Jul 23 13:43:22.437478 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437467 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 23 13:43:22.437693 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437620 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 23 13:43:22.437755 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437726 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 23 13:43:22.437879 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.437832 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 23 13:43:22.438256 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.438234 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 23 13:43:22.438653 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.438636 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 23 13:43:22.438831 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.438816 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 23 13:43:22.439935 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.439878 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 23 13:43:22.442657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.442639 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 23 13:43:22.449932 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.449911 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:43:22.562334 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.562300 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf185aa-ef2b-4e18-a83a-9601c3f1cc20" path="/var/lib/kubelet/pods/8cf185aa-ef2b-4e18-a83a-9601c3f1cc20/volumes" Jul 23 13:43:22.601715 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601691 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601724 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-config\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601744 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601761 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601797 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc9cj\" (UniqueName: \"kubernetes.io/projected/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-kube-api-access-xc9cj\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601895 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601917 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601936 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601953 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.601995 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.601973 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602134 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602024 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602134 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602042 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602134 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602059 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602134 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602081 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602134 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602105 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602134 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602125 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602305 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602163 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602305 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602177 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602305 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602192 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.602305 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.602213 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703148 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703076 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703148 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703110 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703148 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703127 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc9cj\" (UniqueName: \"kubernetes.io/projected/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-kube-api-access-xc9cj\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703405 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703386 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703451 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703427 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703493 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703460 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703493 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703484 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703579 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703490 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703658 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703634 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703809 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703684 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703809 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703709 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703809 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.703809 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703772 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703812 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703904 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703940 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703962 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.703988 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.704016 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704070 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.704046 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704404 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.704085 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-config\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704404 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.704088 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704404 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.704245 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.704760 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.704736 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.706368 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.706339 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.706500 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.706476 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.706744 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-web-config\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707122 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.707054 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707122 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.707069 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707433 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.707341 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707433 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.707357 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707433 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.707388 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-config-out\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.707640 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.707552 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.708041 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.708014 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.708214 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.708197 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.709153 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.709136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.709453 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.709431 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.709576 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.709558 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.709794 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.709778 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-config\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.713156 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.713137 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc9cj\" (UniqueName: \"kubernetes.io/projected/fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f-kube-api-access-xc9cj\") pod \"prometheus-k8s-0\" (UID: \"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.744164 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.744141 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:43:22.885009 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:22.884867 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 23 13:43:22.887442 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:43:22.887416 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa81ddc9_1b21_4540_b18a_5a0fdaf16d6f.slice/crio-1956a62a9c600ce2d13b086238eb524085b47cb95791e63ee819e40e52e3a0b2 WatchSource:0}: Error finding container 1956a62a9c600ce2d13b086238eb524085b47cb95791e63ee819e40e52e3a0b2: Status 404 returned error can't find the container with id 1956a62a9c600ce2d13b086238eb524085b47cb95791e63ee819e40e52e3a0b2 Jul 23 13:43:23.377443 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:23.377408 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f" containerID="a38fc7986c80924e1290a02a773f18240b2e2740846441d4f9199fff09950528" exitCode=0 Jul 23 13:43:23.377858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:23.377481 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerDied","Data":"a38fc7986c80924e1290a02a773f18240b2e2740846441d4f9199fff09950528"} Jul 23 13:43:23.377858 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:23.377537 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"1956a62a9c600ce2d13b086238eb524085b47cb95791e63ee819e40e52e3a0b2"} Jul 23 13:43:24.384349 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.384319 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:43:24.384759 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.384735 2582 generic.go:356] "Generic (PLEG): container finished" podID="fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f" containerID="d60cd52e1d39ddcaa88b1b1d45747fef16cae171342469ae82c2f492ca0e79e5" exitCode=2 Jul 23 13:43:24.384854 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.384817 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"64ca57dd82ca082342daaa660090b63e826efd1ca8e1ad2bd09caf7143c7bd27"} Jul 23 13:43:24.384947 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.384933 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"8fff8da91d2dd2b3815921598ff5daa610ebaa0aca3b8c06e0e3f126dc98c741"} Jul 23 13:43:24.384994 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.384965 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"7259b95c424f480a1d44a4850b58e7af3f50fec48bf7fa3157469c1665f8fdcb"} Jul 23 13:43:24.384994 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.384990 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"d0eed8074221ca63f2eeb75f0a57345ab3bb3d3cd572ff24875baa539d5e1985"} Jul 23 13:43:24.385061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.385015 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"86f91b1425bd1d72ba1c572671d125d36cc03d5b2a5d646d42a963416b3a3d98"} Jul 23 13:43:24.385061 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.385038 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerDied","Data":"d60cd52e1d39ddcaa88b1b1d45747fef16cae171342469ae82c2f492ca0e79e5"} Jul 23 13:43:24.385319 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:24.385308 2582 scope.go:117] "RemoveContainer" containerID="d60cd52e1d39ddcaa88b1b1d45747fef16cae171342469ae82c2f492ca0e79e5" Jul 23 13:43:25.391387 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:25.391360 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:43:25.391863 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:25.391759 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f","Type":"ContainerStarted","Data":"706ae022098392df867ccc157799c069ba31d8abe010812a93ef67a7d61a8005"} Jul 23 13:43:25.432148 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:25.432099 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.413897798 podStartE2EDuration="3.432084893s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.187095ms" isStatefulPod=false podCreationTimestamp="2026-07-23 13:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:43:25.430804627 +0000 UTC m=+157.452516529" watchObservedRunningTime="2026-07-23 13:43:25.432084893 +0000 UTC m=+157.453796773" Jul 23 13:43:27.745028 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:43:27.744985 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:44:22.744296 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:44:22.744263 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:44:22.759015 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:44:22.758991 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:44:23.611416 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:44:23.611390 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 23 13:45:48.351107 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:45:48.351075 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:45:48.357428 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:45:48.357404 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:45:48.364528 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:45:48.364500 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:45:48.367042 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:45:48.367016 2582 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 13:45:48.370573 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:45:48.370555 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:46:11.950577 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.950547 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6"] Jul 23 13:46:11.952727 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.952713 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:11.955780 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.955761 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 13:46:11.956899 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.956884 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lzfws\"" Jul 23 13:46:11.956975 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.956891 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 13:46:11.965907 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.965886 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6"] Jul 23 13:46:11.989782 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.989761 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2hzn\" (UniqueName: \"kubernetes.io/projected/cc34bacd-5074-48d5-8767-35c58abb1ff5-kube-api-access-r2hzn\") pod \"cert-manager-cainjector-559cdd8c87-bbcv6\" (UID: \"cc34bacd-5074-48d5-8767-35c58abb1ff5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:11.989901 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:11.989822 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc34bacd-5074-48d5-8767-35c58abb1ff5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bbcv6\" (UID: \"cc34bacd-5074-48d5-8767-35c58abb1ff5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:12.090190 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.090163 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc34bacd-5074-48d5-8767-35c58abb1ff5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bbcv6\" (UID: \"cc34bacd-5074-48d5-8767-35c58abb1ff5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:12.090298 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.090211 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2hzn\" (UniqueName: \"kubernetes.io/projected/cc34bacd-5074-48d5-8767-35c58abb1ff5-kube-api-access-r2hzn\") pod \"cert-manager-cainjector-559cdd8c87-bbcv6\" (UID: \"cc34bacd-5074-48d5-8767-35c58abb1ff5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:12.097988 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.097958 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc34bacd-5074-48d5-8767-35c58abb1ff5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-bbcv6\" (UID: \"cc34bacd-5074-48d5-8767-35c58abb1ff5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:12.098468 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.098451 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2hzn\" (UniqueName: \"kubernetes.io/projected/cc34bacd-5074-48d5-8767-35c58abb1ff5-kube-api-access-r2hzn\") pod \"cert-manager-cainjector-559cdd8c87-bbcv6\" (UID: \"cc34bacd-5074-48d5-8767-35c58abb1ff5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:12.269753 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.269667 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" Jul 23 13:46:12.391639 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.391596 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6"] Jul 23 13:46:12.395531 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:46:12.395504 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc34bacd_5074_48d5_8767_35c58abb1ff5.slice/crio-b8ce501f93023bad6f3b23a97cc2cb775fcccdc0dd068b74d3b3da3513a1dd07 WatchSource:0}: Error finding container b8ce501f93023bad6f3b23a97cc2cb775fcccdc0dd068b74d3b3da3513a1dd07: Status 404 returned error can't find the container with id b8ce501f93023bad6f3b23a97cc2cb775fcccdc0dd068b74d3b3da3513a1dd07 Jul 23 13:46:12.397406 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.397391 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:46:12.982888 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:12.982838 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" event={"ID":"cc34bacd-5074-48d5-8767-35c58abb1ff5","Type":"ContainerStarted","Data":"b8ce501f93023bad6f3b23a97cc2cb775fcccdc0dd068b74d3b3da3513a1dd07"} Jul 23 13:46:15.996768 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:15.996730 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" event={"ID":"cc34bacd-5074-48d5-8767-35c58abb1ff5","Type":"ContainerStarted","Data":"d7c02ba216ac383487059bb98d3e24baa212492825a126b538365e6c0f95651c"} Jul 23 13:46:16.012475 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:16.012427 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-bbcv6" podStartSLOduration=2.060319997 podStartE2EDuration="5.012410662s" totalImagesPullingTime="2.952090665s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:16.012096761 +0000 UTC m=+328.033808662" watchObservedRunningTime="2026-07-23 13:46:16.012410662 +0000 UTC m=+328.034122541" Jul 23 13:46:22.222911 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.222879 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5slt8"] Jul 23 13:46:22.225049 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.225032 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.227330 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.227309 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dw4sz\"" Jul 23 13:46:22.234155 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.234134 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5slt8"] Jul 23 13:46:22.273541 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.273509 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxjgj\" (UniqueName: \"kubernetes.io/projected/3e73c66c-63d4-463d-9fa7-761f1d0b0a57-kube-api-access-qxjgj\") pod \"cert-manager-55b4d954d5-5slt8\" (UID: \"3e73c66c-63d4-463d-9fa7-761f1d0b0a57\") " pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.273686 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.273564 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e73c66c-63d4-463d-9fa7-761f1d0b0a57-bound-sa-token\") pod \"cert-manager-55b4d954d5-5slt8\" (UID: \"3e73c66c-63d4-463d-9fa7-761f1d0b0a57\") " pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.374776 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.374743 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e73c66c-63d4-463d-9fa7-761f1d0b0a57-bound-sa-token\") pod \"cert-manager-55b4d954d5-5slt8\" (UID: \"3e73c66c-63d4-463d-9fa7-761f1d0b0a57\") " pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.374904 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.374809 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxjgj\" (UniqueName: \"kubernetes.io/projected/3e73c66c-63d4-463d-9fa7-761f1d0b0a57-kube-api-access-qxjgj\") pod \"cert-manager-55b4d954d5-5slt8\" (UID: \"3e73c66c-63d4-463d-9fa7-761f1d0b0a57\") " pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.382692 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.382662 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxjgj\" (UniqueName: \"kubernetes.io/projected/3e73c66c-63d4-463d-9fa7-761f1d0b0a57-kube-api-access-qxjgj\") pod \"cert-manager-55b4d954d5-5slt8\" (UID: \"3e73c66c-63d4-463d-9fa7-761f1d0b0a57\") " pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.382793 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.382744 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3e73c66c-63d4-463d-9fa7-761f1d0b0a57-bound-sa-token\") pod \"cert-manager-55b4d954d5-5slt8\" (UID: \"3e73c66c-63d4-463d-9fa7-761f1d0b0a57\") " pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.533766 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.533677 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5slt8" Jul 23 13:46:22.659353 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:22.659324 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5slt8"] Jul 23 13:46:22.662935 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:46:22.662903 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e73c66c_63d4_463d_9fa7_761f1d0b0a57.slice/crio-243f8ec2c66d9344c740efa73a3666c4c99d52636a59f85c9c15ff12126f0a97 WatchSource:0}: Error finding container 243f8ec2c66d9344c740efa73a3666c4c99d52636a59f85c9c15ff12126f0a97: Status 404 returned error can't find the container with id 243f8ec2c66d9344c740efa73a3666c4c99d52636a59f85c9c15ff12126f0a97 Jul 23 13:46:23.028796 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:23.028727 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5slt8" event={"ID":"3e73c66c-63d4-463d-9fa7-761f1d0b0a57","Type":"ContainerStarted","Data":"913b1e1c87428529fd1e82c827e56240774461482f096a51fad0e0bafbab24d2"} Jul 23 13:46:23.028796 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:23.028805 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5slt8" event={"ID":"3e73c66c-63d4-463d-9fa7-761f1d0b0a57","Type":"ContainerStarted","Data":"243f8ec2c66d9344c740efa73a3666c4c99d52636a59f85c9c15ff12126f0a97"} Jul 23 13:46:23.050684 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:23.050631 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5slt8" podStartSLOduration=1.050615194 podStartE2EDuration="1.050615194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:23.049393018 +0000 UTC m=+335.071104896" watchObservedRunningTime="2026-07-23 13:46:23.050615194 +0000 UTC m=+335.072327073" Jul 23 13:46:43.947806 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.947752 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4"] Jul 23 13:46:43.950992 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.950976 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:43.954386 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.954358 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 13:46:43.955508 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.955488 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 13:46:43.955658 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.955535 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 13:46:43.955658 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.955562 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 13:46:43.955658 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.955567 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 13:46:43.955658 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.955624 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b2bjn\"" Jul 23 13:46:43.961724 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:43.961705 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4"] Jul 23 13:46:44.047822 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.047783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prr44\" (UniqueName: \"kubernetes.io/projected/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-kube-api-access-prr44\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.048050 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.047876 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-cert\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.048050 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.047912 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-manager-config\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.048050 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.047935 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-metrics-cert\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.149242 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.149207 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prr44\" (UniqueName: \"kubernetes.io/projected/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-kube-api-access-prr44\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.149242 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.149242 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-cert\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.149457 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.149260 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-manager-config\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.149457 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.149277 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-metrics-cert\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.149979 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.149952 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-manager-config\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.151602 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.151580 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-cert\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.151692 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.151610 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-metrics-cert\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.159496 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.159471 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prr44\" (UniqueName: \"kubernetes.io/projected/d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b-kube-api-access-prr44\") pod \"lws-controller-manager-6d8d9c69b6-dfkr4\" (UID: \"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b\") " pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.260963 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.260881 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:44.411863 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:44.411810 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4"] Jul 23 13:46:44.414662 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:46:44.414634 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5b7e999_798c_4a5c_bdf3_fe5fa64ba65b.slice/crio-7738597adeec96db09daa5fa5de48de2921b0099b254b12b994420f208a159c9 WatchSource:0}: Error finding container 7738597adeec96db09daa5fa5de48de2921b0099b254b12b994420f208a159c9: Status 404 returned error can't find the container with id 7738597adeec96db09daa5fa5de48de2921b0099b254b12b994420f208a159c9 Jul 23 13:46:45.110750 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:45.110669 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" event={"ID":"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b","Type":"ContainerStarted","Data":"7738597adeec96db09daa5fa5de48de2921b0099b254b12b994420f208a159c9"} Jul 23 13:46:47.119224 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.119191 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" event={"ID":"d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b","Type":"ContainerStarted","Data":"30ff5933abaae62ca7d1143244a001b51491934e19395165983bfc7ab8a2f0ba"} Jul 23 13:46:47.119616 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.119334 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:46:47.147781 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.147735 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" podStartSLOduration=1.911364908 podStartE2EDuration="4.1477199s" totalImagesPullingTime="2.236354992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:47.147012643 +0000 UTC m=+359.168724542" watchObservedRunningTime="2026-07-23 13:46:47.1477199 +0000 UTC m=+359.169431778" Jul 23 13:46:47.534027 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.533926 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz"] Jul 23 13:46:47.537265 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.537234 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.541408 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.541389 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 13:46:47.541936 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.541871 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 13:46:47.542161 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.542140 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 13:46:47.544373 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.542904 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4qrml\"" Jul 23 13:46:47.544373 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.543743 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 13:46:47.585617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.585589 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz"] Jul 23 13:46:47.683815 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.683783 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65f63782-f830-4bd1-9131-90523f098744-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.683815 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.683816 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/65f63782-f830-4bd1-9131-90523f098744-manifests\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.684064 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.683922 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65f63782-f830-4bd1-9131-90523f098744-tmp\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.684064 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.683972 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mf8g\" (UniqueName: \"kubernetes.io/projected/65f63782-f830-4bd1-9131-90523f098744-kube-api-access-7mf8g\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.684064 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.684042 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/65f63782-f830-4bd1-9131-90523f098744-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.785313 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.785228 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65f63782-f830-4bd1-9131-90523f098744-tmp\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.785313 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.785273 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mf8g\" (UniqueName: \"kubernetes.io/projected/65f63782-f830-4bd1-9131-90523f098744-kube-api-access-7mf8g\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.785516 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.785315 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/65f63782-f830-4bd1-9131-90523f098744-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.785516 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.785354 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65f63782-f830-4bd1-9131-90523f098744-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.785516 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.785378 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/65f63782-f830-4bd1-9131-90523f098744-manifests\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.785861 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.785825 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/65f63782-f830-4bd1-9131-90523f098744-manifests\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.787471 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.787443 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/65f63782-f830-4bd1-9131-90523f098744-tmp\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.787704 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.787685 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65f63782-f830-4bd1-9131-90523f098744-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.787774 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.787686 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/65f63782-f830-4bd1-9131-90523f098744-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.802788 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.802764 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mf8g\" (UniqueName: \"kubernetes.io/projected/65f63782-f830-4bd1-9131-90523f098744-kube-api-access-7mf8g\") pod \"opendatahub-operator-controller-manager-7c8b88fb69-xtfvz\" (UID: \"65f63782-f830-4bd1-9131-90523f098744\") " pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.849505 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.849479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:47.984657 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:47.984633 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz"] Jul 23 13:46:47.987337 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:46:47.987308 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f63782_f830_4bd1_9131_90523f098744.slice/crio-5f9fb97ce473f996cba5f697070549d07f9424d163e841c6ba39e440b17c35c1 WatchSource:0}: Error finding container 5f9fb97ce473f996cba5f697070549d07f9424d163e841c6ba39e440b17c35c1: Status 404 returned error can't find the container with id 5f9fb97ce473f996cba5f697070549d07f9424d163e841c6ba39e440b17c35c1 Jul 23 13:46:48.124023 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:48.123995 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" event={"ID":"65f63782-f830-4bd1-9131-90523f098744","Type":"ContainerStarted","Data":"5f9fb97ce473f996cba5f697070549d07f9424d163e841c6ba39e440b17c35c1"} Jul 23 13:46:53.146735 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:53.146701 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" event={"ID":"65f63782-f830-4bd1-9131-90523f098744","Type":"ContainerStarted","Data":"e02982e04dc20cbc0afaa69f5ef098ae080058b5670ace0cdfb71841de41c967"} Jul 23 13:46:54.152021 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:54.151986 2582 generic.go:356] "Generic (PLEG): container finished" podID="65f63782-f830-4bd1-9131-90523f098744" containerID="e02982e04dc20cbc0afaa69f5ef098ae080058b5670ace0cdfb71841de41c967" exitCode=0 Jul 23 13:46:54.152543 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:54.152065 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" event={"ID":"65f63782-f830-4bd1-9131-90523f098744","Type":"ContainerDied","Data":"e02982e04dc20cbc0afaa69f5ef098ae080058b5670ace0cdfb71841de41c967"} Jul 23 13:46:56.161284 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:56.161239 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" event={"ID":"65f63782-f830-4bd1-9131-90523f098744","Type":"ContainerStarted","Data":"15ba8b95056132eaec3c51f65023e956120b3f3d0121276bac14ab81e21850e0"} Jul 23 13:46:56.161671 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:56.161333 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:46:56.187746 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:56.187682 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" podStartSLOduration=2.785191805 podStartE2EDuration="9.187666974s" totalImagesPullingTime="5.394792433s" totalInitContainerRuntime="1.007682736s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:46:47 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:46:56.186192718 +0000 UTC m=+368.207904611" watchObservedRunningTime="2026-07-23 13:46:56.187666974 +0000 UTC m=+368.209378853" Jul 23 13:46:58.126727 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:46:58.126691 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6d8d9c69b6-dfkr4" Jul 23 13:47:07.166998 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:07.166968 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c8b88fb69-xtfvz" Jul 23 13:47:17.952883 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.952831 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw"] Jul 23 13:47:17.955042 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.955026 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:17.961745 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.961723 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 13:47:17.962037 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.962021 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-r7gvl\"" Jul 23 13:47:17.962387 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.962369 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 13:47:17.962745 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.962729 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 13:47:17.962827 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.962762 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 13:47:17.963663 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.963648 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 13:47:17.964341 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:17.964328 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 13:47:18.004271 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.004236 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw"] Jul 23 13:47:18.037383 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037351 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.037383 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037383 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.037578 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.037578 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037477 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4xdb\" (UniqueName: \"kubernetes.io/projected/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-kube-api-access-b4xdb\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.037578 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037508 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.037578 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037527 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.037578 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.037543 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.138903 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.138871 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.138903 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.138901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.139152 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.138918 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.139152 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.138981 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.139152 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.138997 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.139152 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.139024 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.139152 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.139067 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4xdb\" (UniqueName: \"kubernetes.io/projected/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-kube-api-access-b4xdb\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.139807 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.139777 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.141390 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.141362 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.141390 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.141371 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.141559 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.141408 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.141559 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.141524 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.152355 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.152331 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4xdb\" (UniqueName: \"kubernetes.io/projected/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-kube-api-access-b4xdb\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.157087 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.157067 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e7a534b-7337-4f71-8f7d-b99e0ab7704f-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qp5bw\" (UID: \"2e7a534b-7337-4f71-8f7d-b99e0ab7704f\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.264365 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.264270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:18.399280 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:18.399246 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw"] Jul 23 13:47:18.402005 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:47:18.401974 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e7a534b_7337_4f71_8f7d_b99e0ab7704f.slice/crio-579cd4e6f71ed8d9dcec36a55eb0429d08f6846364776f1305c0162cd7be84fd WatchSource:0}: Error finding container 579cd4e6f71ed8d9dcec36a55eb0429d08f6846364776f1305c0162cd7be84fd: Status 404 returned error can't find the container with id 579cd4e6f71ed8d9dcec36a55eb0429d08f6846364776f1305c0162cd7be84fd Jul 23 13:47:19.251368 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:19.251289 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" event={"ID":"2e7a534b-7337-4f71-8f7d-b99e0ab7704f","Type":"ContainerStarted","Data":"579cd4e6f71ed8d9dcec36a55eb0429d08f6846364776f1305c0162cd7be84fd"} Jul 23 13:47:21.328277 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:21.328235 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 13:47:21.328590 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:21.328304 2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 13:47:22.266178 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:22.266143 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" event={"ID":"2e7a534b-7337-4f71-8f7d-b99e0ab7704f","Type":"ContainerStarted","Data":"1f0ff4fc7bec07477dfb8b5cfd7b3b14d54407e17389bddaf60ac7cc1d5b6fa1"} Jul 23 13:47:22.266348 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:22.266299 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:47:22.268045 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:22.268021 2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qp5bw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 13:47:22.268151 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:22.268066 2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" podUID="2e7a534b-7337-4f71-8f7d-b99e0ab7704f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 13:47:22.311095 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:22.311045 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" podStartSLOduration=2.3870184979999998 podStartE2EDuration="5.311030471s" totalImagesPullingTime="2.924011973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:47:22.308365087 +0000 UTC m=+394.330076965" watchObservedRunningTime="2026-07-23 13:47:22.311030471 +0000 UTC m=+394.332742349" Jul 23 13:47:23.275012 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:47:23.274973 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qp5bw" Jul 23 13:48:55.544314 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.544272 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz"] Jul 23 13:48:55.547779 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.547758 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:48:55.550567 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.550528 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 13:48:55.550698 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.550643 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 23 13:48:55.551741 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.551720 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lgqtp\"" Jul 23 13:48:55.551887 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.551813 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 13:48:55.559244 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.559215 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm8x2\" (UniqueName: \"kubernetes.io/projected/a68c23de-96ad-40a8-9e3f-5b2b18cde6f4-kube-api-access-vm8x2\") pod \"dns-operator-controller-manager-864dd78c4c-w92bz\" (UID: \"a68c23de-96ad-40a8-9e3f-5b2b18cde6f4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:48:55.559343 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.559309 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz"] Jul 23 13:48:55.659737 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.659686 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm8x2\" (UniqueName: \"kubernetes.io/projected/a68c23de-96ad-40a8-9e3f-5b2b18cde6f4-kube-api-access-vm8x2\") pod \"dns-operator-controller-manager-864dd78c4c-w92bz\" (UID: \"a68c23de-96ad-40a8-9e3f-5b2b18cde6f4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:48:55.668833 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.668797 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm8x2\" (UniqueName: \"kubernetes.io/projected/a68c23de-96ad-40a8-9e3f-5b2b18cde6f4-kube-api-access-vm8x2\") pod \"dns-operator-controller-manager-864dd78c4c-w92bz\" (UID: \"a68c23de-96ad-40a8-9e3f-5b2b18cde6f4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:48:55.858961 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.858862 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:48:55.990582 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:55.990526 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz"] Jul 23 13:48:55.992949 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:48:55.992919 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda68c23de_96ad_40a8_9e3f_5b2b18cde6f4.slice/crio-0e4f82a29bfecbd6bc84bf7aa7c7f405a782d8824c53cd330e48a2c3a542ef28 WatchSource:0}: Error finding container 0e4f82a29bfecbd6bc84bf7aa7c7f405a782d8824c53cd330e48a2c3a542ef28: Status 404 returned error can't find the container with id 0e4f82a29bfecbd6bc84bf7aa7c7f405a782d8824c53cd330e48a2c3a542ef28 Jul 23 13:48:56.665421 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:56.665389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" event={"ID":"a68c23de-96ad-40a8-9e3f-5b2b18cde6f4","Type":"ContainerStarted","Data":"0e4f82a29bfecbd6bc84bf7aa7c7f405a782d8824c53cd330e48a2c3a542ef28"} Jul 23 13:48:58.676021 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:58.675972 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" event={"ID":"a68c23de-96ad-40a8-9e3f-5b2b18cde6f4","Type":"ContainerStarted","Data":"e5b465a0075bab54b1ff153f4461a78341ceb85a6d26f400f7359d922b9652eb"} Jul 23 13:48:58.676462 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:58.676114 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:48:58.700401 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:48:58.700350 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" podStartSLOduration=1.843872218 podStartE2EDuration="3.700336241s" totalImagesPullingTime="1.856464023s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:48:58.699543401 +0000 UTC m=+490.721255298" watchObservedRunningTime="2026-07-23 13:48:58.700336241 +0000 UTC m=+490.722048211" Jul 23 13:49:06.316031 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.315995 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678"] Jul 23 13:49:06.319272 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.319256 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.322671 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.322645 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p79b2\"" Jul 23 13:49:06.336762 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.336734 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678"] Jul 23 13:49:06.353576 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.353546 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35dc4650-4603-458f-a91d-d14fec0e735a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2c678\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.353576 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.353580 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffgph\" (UniqueName: \"kubernetes.io/projected/35dc4650-4603-458f-a91d-d14fec0e735a-kube-api-access-ffgph\") pod \"kuadrant-operator-controller-manager-86878c769f-2c678\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.455051 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.455012 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35dc4650-4603-458f-a91d-d14fec0e735a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2c678\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.455051 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.455052 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffgph\" (UniqueName: \"kubernetes.io/projected/35dc4650-4603-458f-a91d-d14fec0e735a-kube-api-access-ffgph\") pod \"kuadrant-operator-controller-manager-86878c769f-2c678\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.455482 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.455464 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35dc4650-4603-458f-a91d-d14fec0e735a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-2c678\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.472350 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.472321 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffgph\" (UniqueName: \"kubernetes.io/projected/35dc4650-4603-458f-a91d-d14fec0e735a-kube-api-access-ffgph\") pod \"kuadrant-operator-controller-manager-86878c769f-2c678\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.478104 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.478069 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678"] Jul 23 13:49:06.478359 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.478345 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:06.495593 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.495569 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678"] Jul 23 13:49:06.520784 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.520747 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p"] Jul 23 13:49:06.524101 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.524085 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.555617 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.555576 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-876jq\" (UniqueName: \"kubernetes.io/projected/cd7a749b-c5fc-424d-8a3d-c2702def7c22-kube-api-access-876jq\") pod \"kuadrant-operator-controller-manager-86878c769f-pht7p\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.555823 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.555645 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd7a749b-c5fc-424d-8a3d-c2702def7c22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pht7p\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.557194 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.557165 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p"] Jul 23 13:49:06.656508 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.656472 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd7a749b-c5fc-424d-8a3d-c2702def7c22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pht7p\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.656681 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.656591 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-876jq\" (UniqueName: \"kubernetes.io/projected/cd7a749b-c5fc-424d-8a3d-c2702def7c22-kube-api-access-876jq\") pod \"kuadrant-operator-controller-manager-86878c769f-pht7p\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.656934 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.656908 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd7a749b-c5fc-424d-8a3d-c2702def7c22-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-pht7p\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.672184 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.672160 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-876jq\" (UniqueName: \"kubernetes.io/projected/cd7a749b-c5fc-424d-8a3d-c2702def7c22-kube-api-access-876jq\") pod \"kuadrant-operator-controller-manager-86878c769f-pht7p\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.834431 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.834397 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:06.974421 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:06.974243 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p"] Jul 23 13:49:06.976962 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:49:06.976933 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7a749b_c5fc_424d_8a3d_c2702def7c22.slice/crio-301fc6ef5404e9f300e35b24b17399404fde0d6262fbb0cdf03f231cb53d01bc WatchSource:0}: Error finding container 301fc6ef5404e9f300e35b24b17399404fde0d6262fbb0cdf03f231cb53d01bc: Status 404 returned error can't find the container with id 301fc6ef5404e9f300e35b24b17399404fde0d6262fbb0cdf03f231cb53d01bc Jul 23 13:49:07.714895 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:07.714857 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" event={"ID":"cd7a749b-c5fc-424d-8a3d-c2702def7c22","Type":"ContainerStarted","Data":"301fc6ef5404e9f300e35b24b17399404fde0d6262fbb0cdf03f231cb53d01bc"} Jul 23 13:49:09.024414 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:09.024349 2582 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-2c678_kuadrant-system_35dc4650-4603-458f-a91d-d14fec0e735a_0(cfaea23f21bb4b4af284f1c5ed77a9f99d4091328264afe1a39caf01e7b3ff1a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2c678 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 23 13:49:09.024788 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:09.024437 2582 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-86878c769f-2c678_kuadrant-system_35dc4650-4603-458f-a91d-d14fec0e735a_0(cfaea23f21bb4b4af284f1c5ed77a9f99d4091328264afe1a39caf01e7b3ff1a): error adding pod kuadrant-system_kuadrant-operator-controller-manager-86878c769f-2c678 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:09.682586 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.682554 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w92bz" Jul 23 13:49:09.729262 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.729227 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:09.738487 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.738456 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:09.748791 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:09.748622 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-2c678\" is forbidden: User \"system:node:ip-10-0-128-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-225.ec2.internal' and this object" podUID="35dc4650-4603-458f-a91d-d14fec0e735a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:09.751465 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:09.751425 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-2c678\" is forbidden: User \"system:node:ip-10-0-128-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-225.ec2.internal' and this object" podUID="35dc4650-4603-458f-a91d-d14fec0e735a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:09.783769 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.783734 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35dc4650-4603-458f-a91d-d14fec0e735a-extensions-socket-volume\") pod \"35dc4650-4603-458f-a91d-d14fec0e735a\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " Jul 23 13:49:09.783965 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.783796 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffgph\" (UniqueName: \"kubernetes.io/projected/35dc4650-4603-458f-a91d-d14fec0e735a-kube-api-access-ffgph\") pod \"35dc4650-4603-458f-a91d-d14fec0e735a\" (UID: \"35dc4650-4603-458f-a91d-d14fec0e735a\") " Jul 23 13:49:09.784050 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.784023 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35dc4650-4603-458f-a91d-d14fec0e735a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "35dc4650-4603-458f-a91d-d14fec0e735a" (UID: "35dc4650-4603-458f-a91d-d14fec0e735a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 13:49:09.784137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.784122 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/35dc4650-4603-458f-a91d-d14fec0e735a-extensions-socket-volume\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:49:09.785804 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.785779 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35dc4650-4603-458f-a91d-d14fec0e735a-kube-api-access-ffgph" (OuterVolumeSpecName: "kube-api-access-ffgph") pod "35dc4650-4603-458f-a91d-d14fec0e735a" (UID: "35dc4650-4603-458f-a91d-d14fec0e735a"). InnerVolumeSpecName "kube-api-access-ffgph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 13:49:09.884698 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:09.884664 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffgph\" (UniqueName: \"kubernetes.io/projected/35dc4650-4603-458f-a91d-d14fec0e735a-kube-api-access-ffgph\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 13:49:10.563926 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:10.563879 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35dc4650-4603-458f-a91d-d14fec0e735a" path="/var/lib/kubelet/pods/35dc4650-4603-458f-a91d-d14fec0e735a/volumes" Jul 23 13:49:10.733670 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:10.733631 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:10.736302 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:10.736258 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-2c678\" is forbidden: User \"system:node:ip-10-0-128-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-225.ec2.internal' and this object" podUID="35dc4650-4603-458f-a91d-d14fec0e735a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:10.738481 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:10.738452 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-2c678\" is forbidden: User \"system:node:ip-10-0-128-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-225.ec2.internal' and this object" podUID="35dc4650-4603-458f-a91d-d14fec0e735a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:11.739169 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:11.739136 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" event={"ID":"cd7a749b-c5fc-424d-8a3d-c2702def7c22","Type":"ContainerStarted","Data":"a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314"} Jul 23 13:49:11.739644 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:11.739256 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:49:11.764550 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:11.764501 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" podStartSLOduration=1.613211229 podStartE2EDuration="5.764485272s" totalImagesPullingTime="4.151274043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:49:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:49:11.762832682 +0000 UTC m=+503.784544561" watchObservedRunningTime="2026-07-23 13:49:11.764485272 +0000 UTC m=+503.786197150" Jul 23 13:49:11.765270 ip-10-0-128-225 kubenswrapper[2582]: E0723 13:49:11.765237 2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-86878c769f-2c678\" is forbidden: User \"system:node:ip-10-0-128-225.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-225.ec2.internal' and this object" podUID="35dc4650-4603-458f-a91d-d14fec0e735a" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-2c678" Jul 23 13:49:22.746894 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:49:22.746861 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 13:50:48.379343 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:48.379312 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:50:48.381804 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:48.381782 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:50:48.394418 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:48.394396 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:50:48.397292 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:48.397270 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:50:59.524244 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.524208 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wx2xx"] Jul 23 13:50:59.526492 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.526470 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.529560 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.529537 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 13:50:59.529698 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.529608 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-bgw9w\"" Jul 23 13:50:59.539575 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.539550 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wx2xx"] Jul 23 13:50:59.645913 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.645875 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad450e58-ddf8-4e57-bfc2-25c950b68509-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wx2xx\" (UID: \"ad450e58-ddf8-4e57-bfc2-25c950b68509\") " pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.645913 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.645916 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6cgb\" (UniqueName: \"kubernetes.io/projected/ad450e58-ddf8-4e57-bfc2-25c950b68509-kube-api-access-h6cgb\") pod \"payload-processing-5bf47cd988-wx2xx\" (UID: \"ad450e58-ddf8-4e57-bfc2-25c950b68509\") " pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.746704 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.746665 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad450e58-ddf8-4e57-bfc2-25c950b68509-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wx2xx\" (UID: \"ad450e58-ddf8-4e57-bfc2-25c950b68509\") " pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.746899 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.746712 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6cgb\" (UniqueName: \"kubernetes.io/projected/ad450e58-ddf8-4e57-bfc2-25c950b68509-kube-api-access-h6cgb\") pod \"payload-processing-5bf47cd988-wx2xx\" (UID: \"ad450e58-ddf8-4e57-bfc2-25c950b68509\") " pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.747592 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.747561 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad450e58-ddf8-4e57-bfc2-25c950b68509-plugins-config-volume\") pod \"payload-processing-5bf47cd988-wx2xx\" (UID: \"ad450e58-ddf8-4e57-bfc2-25c950b68509\") " pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.754723 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.754696 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6cgb\" (UniqueName: \"kubernetes.io/projected/ad450e58-ddf8-4e57-bfc2-25c950b68509-kube-api-access-h6cgb\") pod \"payload-processing-5bf47cd988-wx2xx\" (UID: \"ad450e58-ddf8-4e57-bfc2-25c950b68509\") " pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.838256 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.838161 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:50:59.973472 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:50:59.973445 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-wx2xx"] Jul 23 13:50:59.976936 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:50:59.976910 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad450e58_ddf8_4e57_bfc2_25c950b68509.slice/crio-c3398ebd1643b30f0b05743f18972ea6a9fe3b6fca1afc2a0b81c066512b34a5 WatchSource:0}: Error finding container c3398ebd1643b30f0b05743f18972ea6a9fe3b6fca1afc2a0b81c066512b34a5: Status 404 returned error can't find the container with id c3398ebd1643b30f0b05743f18972ea6a9fe3b6fca1afc2a0b81c066512b34a5 Jul 23 13:51:00.210041 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:00.210001 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" event={"ID":"ad450e58-ddf8-4e57-bfc2-25c950b68509","Type":"ContainerStarted","Data":"c3398ebd1643b30f0b05743f18972ea6a9fe3b6fca1afc2a0b81c066512b34a5"} Jul 23 13:51:02.220421 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:02.220385 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" event={"ID":"ad450e58-ddf8-4e57-bfc2-25c950b68509","Type":"ContainerStarted","Data":"e142516e2e3a7908cbf7d86bf3d3afda27a4707ba20c665e5c99e694e51cb6db"} Jul 23 13:51:02.220897 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:02.220456 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:51:02.239596 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:02.239546 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" podStartSLOduration=1.7904808760000002 podStartE2EDuration="3.239527859s" totalImagesPullingTime="1.449046983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:51:02.238024417 +0000 UTC m=+614.259736297" watchObservedRunningTime="2026-07-23 13:51:02.239527859 +0000 UTC m=+614.261239741" Jul 23 13:51:13.227008 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:13.226969 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-wx2xx" Jul 23 13:51:47.097011 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.096980 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz"] Jul 23 13:51:47.099793 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.099774 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.103137 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.103117 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-djm7s\"" Jul 23 13:51:47.104357 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.104327 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 13:51:47.104357 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.104345 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 13:51:47.104520 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.104385 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 13:51:47.115944 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.115890 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz"] Jul 23 13:51:47.260520 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.260490 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.260699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.260528 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljxp\" (UniqueName: \"kubernetes.io/projected/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-kube-api-access-vljxp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.260699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.260553 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.260699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.260571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.260699 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.260659 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.260929 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.260742 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.361728 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.361640 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.361728 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.361682 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljxp\" (UniqueName: \"kubernetes.io/projected/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-kube-api-access-vljxp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.361987 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.361813 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.361987 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.361864 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.361987 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.361912 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.361987 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.361975 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.362239 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.362219 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.362298 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.362254 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.362353 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.362327 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.364029 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.363998 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.364472 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.364448 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.370624 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.370594 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljxp\" (UniqueName: \"kubernetes.io/projected/b8fd398e-3f9c-43b3-8c86-313ae302b4bc-kube-api-access-vljxp\") pod \"e2e-trlp-test-simulated-kserve-598466548d-dklfz\" (UID: \"b8fd398e-3f9c-43b3-8c86-313ae302b4bc\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.410507 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.410479 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:47.544010 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.543983 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz"] Jul 23 13:51:47.546819 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:51:47.546783 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8fd398e_3f9c_43b3_8c86_313ae302b4bc.slice/crio-debc8a62c093992fe7631a5179cd46523e59e29b94a4c315ea5076e725cb1ffc WatchSource:0}: Error finding container debc8a62c093992fe7631a5179cd46523e59e29b94a4c315ea5076e725cb1ffc: Status 404 returned error can't find the container with id debc8a62c093992fe7631a5179cd46523e59e29b94a4c315ea5076e725cb1ffc Jul 23 13:51:47.548190 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:47.548172 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 13:51:48.423694 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:48.423660 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" event={"ID":"b8fd398e-3f9c-43b3-8c86-313ae302b4bc","Type":"ContainerStarted","Data":"debc8a62c093992fe7631a5179cd46523e59e29b94a4c315ea5076e725cb1ffc"} Jul 23 13:51:49.429787 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:49.429742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" event={"ID":"b8fd398e-3f9c-43b3-8c86-313ae302b4bc","Type":"ContainerStarted","Data":"02266591efd29a50b031d41b6acf0164cf11614ec2c29bb782e4e53596bbd0cc"} Jul 23 13:51:49.450283 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:49.450225 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" podStartSLOduration=1.215799729 podStartE2EDuration="2.450203174s" totalImagesPullingTime="1.234403445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:51:49.448996899 +0000 UTC m=+661.470708800" watchObservedRunningTime="2026-07-23 13:51:49.450203174 +0000 UTC m=+661.471915054" Jul 23 13:51:57.411392 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:57.411348 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:57.412003 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:57.411418 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:57.423897 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:57.423875 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:51:57.479012 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:51:57.478986 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-dklfz" Jul 23 13:52:07.981400 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:07.981356 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc"] Jul 23 13:52:07.984056 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:07.984036 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:07.986929 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:07.986910 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 13:52:07.999386 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:07.999364 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc"] Jul 23 13:52:08.036274 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.036251 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.036391 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.036281 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.036391 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.036306 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.036391 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.036335 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svwjt\" (UniqueName: \"kubernetes.io/projected/95dca6a3-790e-45e8-a36e-ff4eada8b972-kube-api-access-svwjt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.036391 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.036361 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.036391 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.036383 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95dca6a3-790e-45e8-a36e-ff4eada8b972-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137107 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137068 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137270 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137119 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svwjt\" (UniqueName: \"kubernetes.io/projected/95dca6a3-790e-45e8-a36e-ff4eada8b972-kube-api-access-svwjt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137270 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137169 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137270 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137211 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95dca6a3-790e-45e8-a36e-ff4eada8b972-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137431 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137293 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137431 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137317 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137641 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137618 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137712 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137638 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.137712 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.137671 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.139742 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.139544 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/95dca6a3-790e-45e8-a36e-ff4eada8b972-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.140168 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.140150 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/95dca6a3-790e-45e8-a36e-ff4eada8b972-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.145670 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.145649 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svwjt\" (UniqueName: \"kubernetes.io/projected/95dca6a3-790e-45e8-a36e-ff4eada8b972-kube-api-access-svwjt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-w6bzc\" (UID: \"95dca6a3-790e-45e8-a36e-ff4eada8b972\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.294278 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.294194 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:08.421677 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.421651 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc"] Jul 23 13:52:08.423634 ip-10-0-128-225 kubenswrapper[2582]: W0723 13:52:08.423607 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95dca6a3_790e_45e8_a36e_ff4eada8b972.slice/crio-1b5cb5630da9dd5e607f0c9c0ed3e35ee3bd3f46e92a3b2f91b53087541fa0c4 WatchSource:0}: Error finding container 1b5cb5630da9dd5e607f0c9c0ed3e35ee3bd3f46e92a3b2f91b53087541fa0c4: Status 404 returned error can't find the container with id 1b5cb5630da9dd5e607f0c9c0ed3e35ee3bd3f46e92a3b2f91b53087541fa0c4 Jul 23 13:52:08.516525 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:08.516485 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" event={"ID":"95dca6a3-790e-45e8-a36e-ff4eada8b972","Type":"ContainerStarted","Data":"1b5cb5630da9dd5e607f0c9c0ed3e35ee3bd3f46e92a3b2f91b53087541fa0c4"} Jul 23 13:52:10.526576 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:10.526534 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" event={"ID":"95dca6a3-790e-45e8-a36e-ff4eada8b972","Type":"ContainerStarted","Data":"dd7f6a9679fcb4d3c74c27246bf948c415eac2249be1b7923e6767c17c4b3eff"} Jul 23 13:52:10.546719 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:10.546669 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" podStartSLOduration=2.264885992 podStartE2EDuration="3.546653133s" totalImagesPullingTime="1.281767141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 13:52:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 13:52:10.545323645 +0000 UTC m=+682.567035534" watchObservedRunningTime="2026-07-23 13:52:10.546653133 +0000 UTC m=+682.568365012" Jul 23 13:52:18.295118 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:18.295077 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:18.295557 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:18.295147 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:18.307316 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:18.307286 2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:52:18.579116 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:52:18.579043 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-w6bzc" Jul 23 13:55:48.410344 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:55:48.410317 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:55:48.410900 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:55:48.410415 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 13:55:48.425478 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:55:48.425453 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 13:55:48.425764 ip-10-0-128-225 kubenswrapper[2582]: I0723 13:55:48.425748 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:00:31.150922 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:31.150886 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p"] Jul 23 14:00:31.151500 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:31.151076 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" podUID="cd7a749b-c5fc-424d-8a3d-c2702def7c22" containerName="manager" containerID="cri-o://a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314" gracePeriod=10 Jul 23 14:00:33.901462 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:33.901428 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 14:00:33.970524 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:33.970496 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-876jq\" (UniqueName: \"kubernetes.io/projected/cd7a749b-c5fc-424d-8a3d-c2702def7c22-kube-api-access-876jq\") pod \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " Jul 23 14:00:33.970704 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:33.970548 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd7a749b-c5fc-424d-8a3d-c2702def7c22-extensions-socket-volume\") pod \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\" (UID: \"cd7a749b-c5fc-424d-8a3d-c2702def7c22\") " Jul 23 14:00:33.970981 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:33.970956 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd7a749b-c5fc-424d-8a3d-c2702def7c22-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cd7a749b-c5fc-424d-8a3d-c2702def7c22" (UID: "cd7a749b-c5fc-424d-8a3d-c2702def7c22"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 14:00:33.972731 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:33.972708 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7a749b-c5fc-424d-8a3d-c2702def7c22-kube-api-access-876jq" (OuterVolumeSpecName: "kube-api-access-876jq") pod "cd7a749b-c5fc-424d-8a3d-c2702def7c22" (UID: "cd7a749b-c5fc-424d-8a3d-c2702def7c22"). InnerVolumeSpecName "kube-api-access-876jq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:00:34.071453 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.071418 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-876jq\" (UniqueName: \"kubernetes.io/projected/cd7a749b-c5fc-424d-8a3d-c2702def7c22-kube-api-access-876jq\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:00:34.071453 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.071448 2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd7a749b-c5fc-424d-8a3d-c2702def7c22-extensions-socket-volume\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:00:34.767614 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.767579 2582 generic.go:356] "Generic (PLEG): container finished" podID="cd7a749b-c5fc-424d-8a3d-c2702def7c22" containerID="a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314" exitCode=0 Jul 23 14:00:34.767781 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.767622 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" Jul 23 14:00:34.767781 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.767623 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" event={"ID":"cd7a749b-c5fc-424d-8a3d-c2702def7c22","Type":"ContainerDied","Data":"a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314"} Jul 23 14:00:34.767781 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.767756 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p" event={"ID":"cd7a749b-c5fc-424d-8a3d-c2702def7c22","Type":"ContainerDied","Data":"301fc6ef5404e9f300e35b24b17399404fde0d6262fbb0cdf03f231cb53d01bc"} Jul 23 14:00:34.767908 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.767787 2582 scope.go:117] "RemoveContainer" containerID="a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314" Jul 23 14:00:34.776794 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.776777 2582 scope.go:117] "RemoveContainer" containerID="a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314" Jul 23 14:00:34.777159 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:00:34.777140 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314\": container with ID starting with a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314 not found: ID does not exist" containerID="a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314" Jul 23 14:00:34.777229 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.777210 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314"} err="failed to get container status \"a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314\": rpc error: code = NotFound desc = could not find container \"a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314\": container with ID starting with a7c017765927f1bc734ed3d47f24ac2bd135c8e5bf65b9178376290ae0eac314 not found: ID does not exist" Jul 23 14:00:34.790353 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.790329 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p"] Jul 23 14:00:34.795255 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:34.795237 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-pht7p"] Jul 23 14:00:36.563522 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:36.563490 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7a749b-c5fc-424d-8a3d-c2702def7c22" path="/var/lib/kubelet/pods/cd7a749b-c5fc-424d-8a3d-c2702def7c22/volumes" Jul 23 14:00:48.439163 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:48.439132 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:00:48.440210 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:48.440188 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:00:48.452098 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:48.452076 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:00:48.453251 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:00:48.453235 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:01:32.329992 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.329959 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs"] Jul 23 14:01:32.332413 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.330309 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd7a749b-c5fc-424d-8a3d-c2702def7c22" containerName="manager" Jul 23 14:01:32.332413 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.330321 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7a749b-c5fc-424d-8a3d-c2702def7c22" containerName="manager" Jul 23 14:01:32.332413 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.330377 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd7a749b-c5fc-424d-8a3d-c2702def7c22" containerName="manager" Jul 23 14:01:32.333343 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.333328 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.336427 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.336407 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-p79b2\"" Jul 23 14:01:32.347953 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.347930 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs"] Jul 23 14:01:32.365595 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.365571 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8923431-61e9-44c6-8458-e33f6f0cdf6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f4rrs\" (UID: \"c8923431-61e9-44c6-8458-e33f6f0cdf6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.365695 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.365644 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22x89\" (UniqueName: \"kubernetes.io/projected/c8923431-61e9-44c6-8458-e33f6f0cdf6c-kube-api-access-22x89\") pod \"kuadrant-operator-controller-manager-86878c769f-f4rrs\" (UID: \"c8923431-61e9-44c6-8458-e33f6f0cdf6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.466174 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.466136 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22x89\" (UniqueName: \"kubernetes.io/projected/c8923431-61e9-44c6-8458-e33f6f0cdf6c-kube-api-access-22x89\") pod \"kuadrant-operator-controller-manager-86878c769f-f4rrs\" (UID: \"c8923431-61e9-44c6-8458-e33f6f0cdf6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.466341 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.466203 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8923431-61e9-44c6-8458-e33f6f0cdf6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f4rrs\" (UID: \"c8923431-61e9-44c6-8458-e33f6f0cdf6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.466568 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.466553 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c8923431-61e9-44c6-8458-e33f6f0cdf6c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f4rrs\" (UID: \"c8923431-61e9-44c6-8458-e33f6f0cdf6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.475006 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.474979 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22x89\" (UniqueName: \"kubernetes.io/projected/c8923431-61e9-44c6-8458-e33f6f0cdf6c-kube-api-access-22x89\") pod \"kuadrant-operator-controller-manager-86878c769f-f4rrs\" (UID: \"c8923431-61e9-44c6-8458-e33f6f0cdf6c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.643496 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.643464 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:32.784283 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:32.784100 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs"] Jul 23 14:01:32.787670 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:01:32.787640 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8923431_61e9_44c6_8458_e33f6f0cdf6c.slice/crio-43bd5ba611585ecc4b202af15cd4b5d78254bbb20753a40b16d26c24afe31984 WatchSource:0}: Error finding container 43bd5ba611585ecc4b202af15cd4b5d78254bbb20753a40b16d26c24afe31984: Status 404 returned error can't find the container with id 43bd5ba611585ecc4b202af15cd4b5d78254bbb20753a40b16d26c24afe31984 Jul 23 14:01:33.014006 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:33.013918 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" event={"ID":"c8923431-61e9-44c6-8458-e33f6f0cdf6c","Type":"ContainerStarted","Data":"8d3bd1405c84612977a318e1e83fe6517471b2d75ccc4b659dc11def1df2657f"} Jul 23 14:01:33.014006 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:33.013967 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" event={"ID":"c8923431-61e9-44c6-8458-e33f6f0cdf6c","Type":"ContainerStarted","Data":"43bd5ba611585ecc4b202af15cd4b5d78254bbb20753a40b16d26c24afe31984"} Jul 23 14:01:33.014224 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:33.014018 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:01:33.034135 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:33.034089 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" podStartSLOduration=1.034071465 podStartE2EDuration="1.034071465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:01:33.033470716 +0000 UTC m=+1245.055182595" watchObservedRunningTime="2026-07-23 14:01:33.034071465 +0000 UTC m=+1245.055783343" Jul 23 14:01:44.020894 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:01:44.020838 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f4rrs" Jul 23 14:05:48.465614 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:05:48.465584 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:05:48.467562 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:05:48.467538 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:05:48.478709 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:05:48.478678 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:05:48.480585 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:05:48.480563 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:09:17.152593 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.152521 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd"] Jul 23 14:09:17.156012 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.155997 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.165357 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.165334 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:09:17.165486 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.165443 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:09:17.165569 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.165555 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-cda79bb3-serving-cert\"" Jul 23 14:09:17.166650 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.166634 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:09:17.178172 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.178150 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd"] Jul 23 14:09:17.238853 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.238806 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv7mx\" (UniqueName: \"kubernetes.io/projected/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-kube-api-access-cv7mx\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.239017 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.238880 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.239017 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.238904 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-openshift-service-ca\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.339914 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.339886 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv7mx\" (UniqueName: \"kubernetes.io/projected/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-kube-api-access-cv7mx\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.340069 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.339947 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.340069 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.339971 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-openshift-service-ca\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.340145 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:09:17.340085 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-serving-cert: secret "maas-api-e2e-ait-cda79bb3-serving-cert" not found Jul 23 14:09:17.340179 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:09:17.340151 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls podName:5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01 nodeName:}" failed. No retries permitted until 2026-07-23 14:09:17.84013148 +0000 UTC m=+1709.861843339 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls") pod "maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" (UID: "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01") : secret "maas-api-e2e-ait-cda79bb3-serving-cert" not found Jul 23 14:09:17.340551 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.340533 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-openshift-service-ca\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.356260 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.356229 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv7mx\" (UniqueName: \"kubernetes.io/projected/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-kube-api-access-cv7mx\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.844403 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.844372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:17.846705 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:17.846680 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls\") pod \"maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:18.065669 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:18.065636 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:18.220997 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:18.220920 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd"] Jul 23 14:09:18.223258 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:09:18.223232 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e49e94e_7b2e_4b18_9f65_3ca0d8e9ae01.slice/crio-5cf4620945621e1b08f0b04df637805a3f02d6ab311a2d5585c4854076ab4048 WatchSource:0}: Error finding container 5cf4620945621e1b08f0b04df637805a3f02d6ab311a2d5585c4854076ab4048: Status 404 returned error can't find the container with id 5cf4620945621e1b08f0b04df637805a3f02d6ab311a2d5585c4854076ab4048 Jul 23 14:09:18.224414 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:18.224398 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 14:09:19.071964 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:19.071917 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" event={"ID":"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01","Type":"ContainerStarted","Data":"5cf4620945621e1b08f0b04df637805a3f02d6ab311a2d5585c4854076ab4048"} Jul 23 14:09:21.082828 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:21.082742 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" event={"ID":"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01","Type":"ContainerStarted","Data":"a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2"} Jul 23 14:09:21.083219 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:21.082886 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:21.120681 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:21.120633 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" podStartSLOduration=1.622356621 podStartE2EDuration="4.120619089s" totalImagesPullingTime="2.498262468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:09:21.120410721 +0000 UTC m=+1713.142122600" watchObservedRunningTime="2026-07-23 14:09:21.120619089 +0000 UTC m=+1713.142330997" Jul 23 14:09:27.092355 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:27.092326 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:53.235566 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.235536 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd"] Jul 23 14:09:53.236090 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.235724 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" podUID="5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" containerName="maas-api" containerID="cri-o://a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2" gracePeriod=30 Jul 23 14:09:53.487381 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.487319 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:53.563632 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.563599 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-openshift-service-ca\") pod \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " Jul 23 14:09:53.563894 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.563663 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv7mx\" (UniqueName: \"kubernetes.io/projected/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-kube-api-access-cv7mx\") pod \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " Jul 23 14:09:53.563894 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.563749 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls\") pod \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\" (UID: \"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01\") " Jul 23 14:09:53.564088 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.564016 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" (UID: "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:09:53.566347 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.566313 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-kube-api-access-cv7mx" (OuterVolumeSpecName: "kube-api-access-cv7mx") pod "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" (UID: "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01"). InnerVolumeSpecName "kube-api-access-cv7mx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:09:53.566462 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.566433 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" (UID: "5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:09:53.664793 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.664756 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv7mx\" (UniqueName: \"kubernetes.io/projected/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-kube-api-access-cv7mx\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:09:53.664793 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.664784 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:09:53.664793 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:53.664794 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:09:54.226488 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.226451 2582 generic.go:356] "Generic (PLEG): container finished" podID="5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" containerID="a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2" exitCode=0 Jul 23 14:09:54.226671 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.226496 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" Jul 23 14:09:54.226671 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.226522 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" event={"ID":"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01","Type":"ContainerDied","Data":"a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2"} Jul 23 14:09:54.226671 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.226562 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd" event={"ID":"5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01","Type":"ContainerDied","Data":"5cf4620945621e1b08f0b04df637805a3f02d6ab311a2d5585c4854076ab4048"} Jul 23 14:09:54.226671 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.226583 2582 scope.go:117] "RemoveContainer" containerID="a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2" Jul 23 14:09:54.237152 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.237039 2582 scope.go:117] "RemoveContainer" containerID="a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2" Jul 23 14:09:54.237384 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:09:54.237332 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2\": container with ID starting with a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2 not found: ID does not exist" containerID="a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2" Jul 23 14:09:54.237434 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.237370 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2"} err="failed to get container status \"a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2\": rpc error: code = NotFound desc = could not find container \"a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2\": container with ID starting with a6a26368525bb26f986736c7c5c93342bfb7bf2062e520da8e7a277a0209bcc2 not found: ID does not exist" Jul 23 14:09:54.255266 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.255237 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd"] Jul 23 14:09:54.263514 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.263493 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-cda79bb3-7cc54db7f6-8zltd"] Jul 23 14:09:54.563895 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:09:54.563805 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" path="/var/lib/kubelet/pods/5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01/volumes" Jul 23 14:10:01.489747 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.489717 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8"] Jul 23 14:10:01.490206 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.490087 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" containerName="maas-api" Jul 23 14:10:01.490206 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.490099 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" containerName="maas-api" Jul 23 14:10:01.490206 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.490161 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e49e94e-7b2e-4b18-9f65-3ca0d8e9ae01" containerName="maas-api" Jul 23 14:10:01.496325 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.496307 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.499728 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.499708 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:10:01.499880 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.499707 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d9660af7-serving-cert\"" Jul 23 14:10:01.500714 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.500696 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:10:01.500798 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.500698 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:10:01.510947 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.510925 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8"] Jul 23 14:10:01.637124 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.637090 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.637332 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.637145 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f51b584-aee9-42a2-91e2-537cbf773ffd-openshift-service-ca\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.637332 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.637245 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qk85\" (UniqueName: \"kubernetes.io/projected/8f51b584-aee9-42a2-91e2-537cbf773ffd-kube-api-access-8qk85\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.738494 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.738455 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qk85\" (UniqueName: \"kubernetes.io/projected/8f51b584-aee9-42a2-91e2-537cbf773ffd-kube-api-access-8qk85\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.738712 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.738517 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.738712 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.738558 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f51b584-aee9-42a2-91e2-537cbf773ffd-openshift-service-ca\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.738712 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:10:01.738664 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-serving-cert: secret "maas-api-e2e-migrate-d9660af7-serving-cert" not found Jul 23 14:10:01.738931 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:10:01.738717 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls podName:8f51b584-aee9-42a2-91e2-537cbf773ffd nodeName:}" failed. No retries permitted until 2026-07-23 14:10:02.238702769 +0000 UTC m=+1754.260414626 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls") pod "maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" (UID: "8f51b584-aee9-42a2-91e2-537cbf773ffd") : secret "maas-api-e2e-migrate-d9660af7-serving-cert" not found Jul 23 14:10:01.739168 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.739151 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f51b584-aee9-42a2-91e2-537cbf773ffd-openshift-service-ca\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:01.753175 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:01.753106 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qk85\" (UniqueName: \"kubernetes.io/projected/8f51b584-aee9-42a2-91e2-537cbf773ffd-kube-api-access-8qk85\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:02.243168 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:02.243126 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:02.245613 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:02.245590 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls\") pod \"maas-api-e2e-migrate-d9660af7-544779c689-k4fb8\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:02.406569 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:02.406535 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:02.539161 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:02.539137 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8"] Jul 23 14:10:02.541443 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:10:02.541416 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f51b584_aee9_42a2_91e2_537cbf773ffd.slice/crio-a74748ba84889c781c9f619e46f31fe94831a5b06abdf0cf9466756a2d552b50 WatchSource:0}: Error finding container a74748ba84889c781c9f619e46f31fe94831a5b06abdf0cf9466756a2d552b50: Status 404 returned error can't find the container with id a74748ba84889c781c9f619e46f31fe94831a5b06abdf0cf9466756a2d552b50 Jul 23 14:10:03.268404 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:03.268367 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" event={"ID":"8f51b584-aee9-42a2-91e2-537cbf773ffd","Type":"ContainerStarted","Data":"a74748ba84889c781c9f619e46f31fe94831a5b06abdf0cf9466756a2d552b50"} Jul 23 14:10:04.274215 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:04.274164 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" event={"ID":"8f51b584-aee9-42a2-91e2-537cbf773ffd","Type":"ContainerStarted","Data":"4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba"} Jul 23 14:10:04.274687 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:04.274331 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:04.304467 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:04.304412 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" podStartSLOduration=2.422478298 podStartE2EDuration="3.304395694s" totalImagesPullingTime="881.917396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:10:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:10:04.302512817 +0000 UTC m=+1756.324224708" watchObservedRunningTime="2026-07-23 14:10:04.304395694 +0000 UTC m=+1756.326107616" Jul 23 14:10:10.283459 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:10.283430 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:36.768676 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:36.768644 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8"] Jul 23 14:10:36.769364 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:36.768868 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" podUID="8f51b584-aee9-42a2-91e2-537cbf773ffd" containerName="maas-api" containerID="cri-o://4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba" gracePeriod=30 Jul 23 14:10:37.019224 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.019169 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:37.045556 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.045529 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qk85\" (UniqueName: \"kubernetes.io/projected/8f51b584-aee9-42a2-91e2-537cbf773ffd-kube-api-access-8qk85\") pod \"8f51b584-aee9-42a2-91e2-537cbf773ffd\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " Jul 23 14:10:37.045675 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.045564 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls\") pod \"8f51b584-aee9-42a2-91e2-537cbf773ffd\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " Jul 23 14:10:37.045675 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.045592 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f51b584-aee9-42a2-91e2-537cbf773ffd-openshift-service-ca\") pod \"8f51b584-aee9-42a2-91e2-537cbf773ffd\" (UID: \"8f51b584-aee9-42a2-91e2-537cbf773ffd\") " Jul 23 14:10:37.046024 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.045995 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f51b584-aee9-42a2-91e2-537cbf773ffd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f51b584-aee9-42a2-91e2-537cbf773ffd" (UID: "8f51b584-aee9-42a2-91e2-537cbf773ffd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:10:37.047666 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.047645 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f51b584-aee9-42a2-91e2-537cbf773ffd-kube-api-access-8qk85" (OuterVolumeSpecName: "kube-api-access-8qk85") pod "8f51b584-aee9-42a2-91e2-537cbf773ffd" (UID: "8f51b584-aee9-42a2-91e2-537cbf773ffd"). InnerVolumeSpecName "kube-api-access-8qk85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:10:37.047666 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.047651 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f51b584-aee9-42a2-91e2-537cbf773ffd" (UID: "8f51b584-aee9-42a2-91e2-537cbf773ffd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:10:37.147316 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.147282 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qk85\" (UniqueName: \"kubernetes.io/projected/8f51b584-aee9-42a2-91e2-537cbf773ffd-kube-api-access-8qk85\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:10:37.147316 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.147311 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f51b584-aee9-42a2-91e2-537cbf773ffd-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:10:37.147316 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.147320 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f51b584-aee9-42a2-91e2-537cbf773ffd-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:10:37.416204 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.416170 2582 generic.go:356] "Generic (PLEG): container finished" podID="8f51b584-aee9-42a2-91e2-537cbf773ffd" containerID="4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba" exitCode=0 Jul 23 14:10:37.416381 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.416219 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" Jul 23 14:10:37.416381 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.416263 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" event={"ID":"8f51b584-aee9-42a2-91e2-537cbf773ffd","Type":"ContainerDied","Data":"4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba"} Jul 23 14:10:37.416381 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.416326 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8" event={"ID":"8f51b584-aee9-42a2-91e2-537cbf773ffd","Type":"ContainerDied","Data":"a74748ba84889c781c9f619e46f31fe94831a5b06abdf0cf9466756a2d552b50"} Jul 23 14:10:37.416381 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.416354 2582 scope.go:117] "RemoveContainer" containerID="4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba" Jul 23 14:10:37.425400 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.425384 2582 scope.go:117] "RemoveContainer" containerID="4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba" Jul 23 14:10:37.425642 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:10:37.425624 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba\": container with ID starting with 4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba not found: ID does not exist" containerID="4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba" Jul 23 14:10:37.425711 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.425649 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba"} err="failed to get container status \"4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba\": rpc error: code = NotFound desc = could not find container \"4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba\": container with ID starting with 4d19c3ba8218a4d8cef45eaba24746b6c329daecdf3d5a69c9ec6c752c1bc1ba not found: ID does not exist" Jul 23 14:10:37.437609 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.437582 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8"] Jul 23 14:10:37.440819 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:37.440791 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d9660af7-544779c689-k4fb8"] Jul 23 14:10:38.563276 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:38.563240 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f51b584-aee9-42a2-91e2-537cbf773ffd" path="/var/lib/kubelet/pods/8f51b584-aee9-42a2-91e2-537cbf773ffd/volumes" Jul 23 14:10:44.640324 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.640295 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g"] Jul 23 14:10:44.640674 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.640623 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f51b584-aee9-42a2-91e2-537cbf773ffd" containerName="maas-api" Jul 23 14:10:44.640674 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.640634 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f51b584-aee9-42a2-91e2-537cbf773ffd" containerName="maas-api" Jul 23 14:10:44.640748 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.640682 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f51b584-aee9-42a2-91e2-537cbf773ffd" containerName="maas-api" Jul 23 14:10:44.645071 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.645052 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.647459 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.647435 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:10:44.647563 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.647511 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-daf75dd9-serving-cert\"" Jul 23 14:10:44.648567 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.648552 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:10:44.648627 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.648559 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:10:44.660403 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.660381 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g"] Jul 23 14:10:44.717943 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.717911 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5rqf\" (UniqueName: \"kubernetes.io/projected/b93c7ffa-f325-4063-adee-80fc918fe9b3-kube-api-access-q5rqf\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.718094 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.717948 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.718094 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.718006 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93c7ffa-f325-4063-adee-80fc918fe9b3-openshift-service-ca\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.818866 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.818818 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93c7ffa-f325-4063-adee-80fc918fe9b3-openshift-service-ca\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.819025 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.818915 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5rqf\" (UniqueName: \"kubernetes.io/projected/b93c7ffa-f325-4063-adee-80fc918fe9b3-kube-api-access-q5rqf\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.819025 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.818944 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.819108 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:10:44.819043 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-serving-cert: secret "maas-api-e2e-ait-daf75dd9-serving-cert" not found Jul 23 14:10:44.819108 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:10:44.819106 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls podName:b93c7ffa-f325-4063-adee-80fc918fe9b3 nodeName:}" failed. No retries permitted until 2026-07-23 14:10:45.319090489 +0000 UTC m=+1797.340802345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls") pod "maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" (UID: "b93c7ffa-f325-4063-adee-80fc918fe9b3") : secret "maas-api-e2e-ait-daf75dd9-serving-cert" not found Jul 23 14:10:44.819419 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.819399 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93c7ffa-f325-4063-adee-80fc918fe9b3-openshift-service-ca\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:44.829158 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:44.829129 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5rqf\" (UniqueName: \"kubernetes.io/projected/b93c7ffa-f325-4063-adee-80fc918fe9b3-kube-api-access-q5rqf\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:45.323452 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:45.323421 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:45.325893 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:45.325872 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls\") pod \"maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:45.555535 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:45.555497 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:45.695051 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:45.695028 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g"] Jul 23 14:10:45.697066 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:10:45.697042 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb93c7ffa_f325_4063_adee_80fc918fe9b3.slice/crio-b6389165dc31fd13b464e5e386ce53b745164d738ad02a9ac111df897c150be9 WatchSource:0}: Error finding container b6389165dc31fd13b464e5e386ce53b745164d738ad02a9ac111df897c150be9: Status 404 returned error can't find the container with id b6389165dc31fd13b464e5e386ce53b745164d738ad02a9ac111df897c150be9 Jul 23 14:10:46.460035 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:46.460010 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" event={"ID":"b93c7ffa-f325-4063-adee-80fc918fe9b3","Type":"ContainerStarted","Data":"97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b"} Jul 23 14:10:46.460154 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:46.460064 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" event={"ID":"b93c7ffa-f325-4063-adee-80fc918fe9b3","Type":"ContainerStarted","Data":"b6389165dc31fd13b464e5e386ce53b745164d738ad02a9ac111df897c150be9"} Jul 23 14:10:46.460154 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:46.460112 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:10:46.481801 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:46.481757 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" podStartSLOduration=1.7937988200000001 podStartE2EDuration="2.481743503s" totalImagesPullingTime="687.944683ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:10:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:10:46.479568119 +0000 UTC m=+1798.501280036" watchObservedRunningTime="2026-07-23 14:10:46.481743503 +0000 UTC m=+1798.503455381" Jul 23 14:10:48.491727 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:48.491702 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:10:48.494433 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:48.494414 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:10:48.504335 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:48.504315 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:10:48.506629 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:48.506610 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:10:52.471009 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:10:52.470978 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:11:20.763789 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:20.763756 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g"] Jul 23 14:11:20.764380 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:20.764073 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" podUID="b93c7ffa-f325-4063-adee-80fc918fe9b3" containerName="maas-api" containerID="cri-o://97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b" gracePeriod=30 Jul 23 14:11:21.024884 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.024806 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:11:21.144371 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.144334 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls\") pod \"b93c7ffa-f325-4063-adee-80fc918fe9b3\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " Jul 23 14:11:21.144574 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.144402 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93c7ffa-f325-4063-adee-80fc918fe9b3-openshift-service-ca\") pod \"b93c7ffa-f325-4063-adee-80fc918fe9b3\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " Jul 23 14:11:21.144574 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.144441 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5rqf\" (UniqueName: \"kubernetes.io/projected/b93c7ffa-f325-4063-adee-80fc918fe9b3-kube-api-access-q5rqf\") pod \"b93c7ffa-f325-4063-adee-80fc918fe9b3\" (UID: \"b93c7ffa-f325-4063-adee-80fc918fe9b3\") " Jul 23 14:11:21.144861 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.144816 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b93c7ffa-f325-4063-adee-80fc918fe9b3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b93c7ffa-f325-4063-adee-80fc918fe9b3" (UID: "b93c7ffa-f325-4063-adee-80fc918fe9b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:11:21.146626 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.146600 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93c7ffa-f325-4063-adee-80fc918fe9b3-kube-api-access-q5rqf" (OuterVolumeSpecName: "kube-api-access-q5rqf") pod "b93c7ffa-f325-4063-adee-80fc918fe9b3" (UID: "b93c7ffa-f325-4063-adee-80fc918fe9b3"). InnerVolumeSpecName "kube-api-access-q5rqf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:11:21.146738 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.146624 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b93c7ffa-f325-4063-adee-80fc918fe9b3" (UID: "b93c7ffa-f325-4063-adee-80fc918fe9b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:11:21.245754 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.245715 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b93c7ffa-f325-4063-adee-80fc918fe9b3-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:11:21.245754 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.245746 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b93c7ffa-f325-4063-adee-80fc918fe9b3-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:11:21.245754 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.245757 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5rqf\" (UniqueName: \"kubernetes.io/projected/b93c7ffa-f325-4063-adee-80fc918fe9b3-kube-api-access-q5rqf\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:11:21.615003 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.614968 2582 generic.go:356] "Generic (PLEG): container finished" podID="b93c7ffa-f325-4063-adee-80fc918fe9b3" containerID="97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b" exitCode=0 Jul 23 14:11:21.615210 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.615012 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" event={"ID":"b93c7ffa-f325-4063-adee-80fc918fe9b3","Type":"ContainerDied","Data":"97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b"} Jul 23 14:11:21.615210 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.615016 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" Jul 23 14:11:21.615210 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.615053 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g" event={"ID":"b93c7ffa-f325-4063-adee-80fc918fe9b3","Type":"ContainerDied","Data":"b6389165dc31fd13b464e5e386ce53b745164d738ad02a9ac111df897c150be9"} Jul 23 14:11:21.615210 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.615074 2582 scope.go:117] "RemoveContainer" containerID="97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b" Jul 23 14:11:21.623325 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.623310 2582 scope.go:117] "RemoveContainer" containerID="97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b" Jul 23 14:11:21.623565 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:11:21.623548 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b\": container with ID starting with 97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b not found: ID does not exist" containerID="97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b" Jul 23 14:11:21.623612 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.623571 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b"} err="failed to get container status \"97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b\": rpc error: code = NotFound desc = could not find container \"97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b\": container with ID starting with 97d739e4f0962694eb8cc8c3040eb5494e9395bdb7d6e66b869a4882327d820b not found: ID does not exist" Jul 23 14:11:21.637647 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.637625 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g"] Jul 23 14:11:21.641149 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:21.641114 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-daf75dd9-5dddf5cdfc-87l4g"] Jul 23 14:11:22.562785 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:22.562743 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93c7ffa-f325-4063-adee-80fc918fe9b3" path="/var/lib/kubelet/pods/b93c7ffa-f325-4063-adee-80fc918fe9b3/volumes" Jul 23 14:11:29.980773 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.980745 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx"] Jul 23 14:11:29.981267 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.981093 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93c7ffa-f325-4063-adee-80fc918fe9b3" containerName="maas-api" Jul 23 14:11:29.981267 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.981106 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93c7ffa-f325-4063-adee-80fc918fe9b3" containerName="maas-api" Jul 23 14:11:29.981267 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.981173 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93c7ffa-f325-4063-adee-80fc918fe9b3" containerName="maas-api" Jul 23 14:11:29.985473 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.985457 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:29.988299 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.988254 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:11:29.988424 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.988303 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-854fd7f2-serving-cert\"" Jul 23 14:11:29.989353 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.989331 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:11:29.989466 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.989334 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:11:29.995797 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:29.995767 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx"] Jul 23 14:11:30.131533 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.131496 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v44fn\" (UniqueName: \"kubernetes.io/projected/ab05b437-9f97-4e88-94ee-082082dcd845-kube-api-access-v44fn\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.131711 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.131555 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab05b437-9f97-4e88-94ee-082082dcd845-openshift-service-ca\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.131711 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.131641 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.232763 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.232679 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab05b437-9f97-4e88-94ee-082082dcd845-openshift-service-ca\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.232763 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.232720 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.232999 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.232788 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v44fn\" (UniqueName: \"kubernetes.io/projected/ab05b437-9f97-4e88-94ee-082082dcd845-kube-api-access-v44fn\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.232999 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:11:30.232913 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-serving-cert: secret "maas-api-e2e-derive-854fd7f2-serving-cert" not found Jul 23 14:11:30.232999 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:11:30.232986 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls podName:ab05b437-9f97-4e88-94ee-082082dcd845 nodeName:}" failed. No retries permitted until 2026-07-23 14:11:30.732971176 +0000 UTC m=+1842.754683038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls") pod "maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" (UID: "ab05b437-9f97-4e88-94ee-082082dcd845") : secret "maas-api-e2e-derive-854fd7f2-serving-cert" not found Jul 23 14:11:30.233325 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.233307 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab05b437-9f97-4e88-94ee-082082dcd845-openshift-service-ca\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.242228 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.242202 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v44fn\" (UniqueName: \"kubernetes.io/projected/ab05b437-9f97-4e88-94ee-082082dcd845-kube-api-access-v44fn\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.735992 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.735963 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.738430 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.738407 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls\") pod \"maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:30.896220 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:30.896182 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:31.029731 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:31.029681 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx"] Jul 23 14:11:31.032396 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:11:31.032362 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab05b437_9f97_4e88_94ee_082082dcd845.slice/crio-6bea975fd3b9ccc6cb9db2870bb771964282328febc6f05fce71869c9c962db5 WatchSource:0}: Error finding container 6bea975fd3b9ccc6cb9db2870bb771964282328febc6f05fce71869c9c962db5: Status 404 returned error can't find the container with id 6bea975fd3b9ccc6cb9db2870bb771964282328febc6f05fce71869c9c962db5 Jul 23 14:11:31.661774 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:31.661696 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" event={"ID":"ab05b437-9f97-4e88-94ee-082082dcd845","Type":"ContainerStarted","Data":"f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943"} Jul 23 14:11:31.661774 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:31.661753 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" event={"ID":"ab05b437-9f97-4e88-94ee-082082dcd845","Type":"ContainerStarted","Data":"6bea975fd3b9ccc6cb9db2870bb771964282328febc6f05fce71869c9c962db5"} Jul 23 14:11:31.662020 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:31.661783 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:11:31.683423 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:31.683379 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" podStartSLOduration=2.313213399 podStartE2EDuration="2.683364946s" totalImagesPullingTime="370.151547ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:11:31.681618328 +0000 UTC m=+1843.703330208" watchObservedRunningTime="2026-07-23 14:11:31.683364946 +0000 UTC m=+1843.705076825" Jul 23 14:11:37.671832 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:11:37.671797 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:12:05.257202 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.257164 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx"] Jul 23 14:12:05.257793 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.257424 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" podUID="ab05b437-9f97-4e88-94ee-082082dcd845" containerName="maas-api" containerID="cri-o://f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943" gracePeriod=30 Jul 23 14:12:05.508408 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.508343 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:12:05.661155 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.661126 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab05b437-9f97-4e88-94ee-082082dcd845-openshift-service-ca\") pod \"ab05b437-9f97-4e88-94ee-082082dcd845\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " Jul 23 14:12:05.661155 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.661161 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v44fn\" (UniqueName: \"kubernetes.io/projected/ab05b437-9f97-4e88-94ee-082082dcd845-kube-api-access-v44fn\") pod \"ab05b437-9f97-4e88-94ee-082082dcd845\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " Jul 23 14:12:05.661362 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.661201 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls\") pod \"ab05b437-9f97-4e88-94ee-082082dcd845\" (UID: \"ab05b437-9f97-4e88-94ee-082082dcd845\") " Jul 23 14:12:05.661514 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.661479 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab05b437-9f97-4e88-94ee-082082dcd845-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab05b437-9f97-4e88-94ee-082082dcd845" (UID: "ab05b437-9f97-4e88-94ee-082082dcd845"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:12:05.663257 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.663230 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab05b437-9f97-4e88-94ee-082082dcd845-kube-api-access-v44fn" (OuterVolumeSpecName: "kube-api-access-v44fn") pod "ab05b437-9f97-4e88-94ee-082082dcd845" (UID: "ab05b437-9f97-4e88-94ee-082082dcd845"). InnerVolumeSpecName "kube-api-access-v44fn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:12:05.663365 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.663261 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab05b437-9f97-4e88-94ee-082082dcd845" (UID: "ab05b437-9f97-4e88-94ee-082082dcd845"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:12:05.762544 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.762458 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab05b437-9f97-4e88-94ee-082082dcd845-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:12:05.762544 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.762488 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v44fn\" (UniqueName: \"kubernetes.io/projected/ab05b437-9f97-4e88-94ee-082082dcd845-kube-api-access-v44fn\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:12:05.762544 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.762499 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab05b437-9f97-4e88-94ee-082082dcd845-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:12:05.807715 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.807683 2582 generic.go:356] "Generic (PLEG): container finished" podID="ab05b437-9f97-4e88-94ee-082082dcd845" containerID="f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943" exitCode=0 Jul 23 14:12:05.807901 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.807733 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" Jul 23 14:12:05.807901 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.807773 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" event={"ID":"ab05b437-9f97-4e88-94ee-082082dcd845","Type":"ContainerDied","Data":"f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943"} Jul 23 14:12:05.807901 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.807827 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx" event={"ID":"ab05b437-9f97-4e88-94ee-082082dcd845","Type":"ContainerDied","Data":"6bea975fd3b9ccc6cb9db2870bb771964282328febc6f05fce71869c9c962db5"} Jul 23 14:12:05.807901 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.807878 2582 scope.go:117] "RemoveContainer" containerID="f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943" Jul 23 14:12:05.816891 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.816826 2582 scope.go:117] "RemoveContainer" containerID="f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943" Jul 23 14:12:05.817126 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:12:05.817107 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943\": container with ID starting with f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943 not found: ID does not exist" containerID="f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943" Jul 23 14:12:05.817185 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.817136 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943"} err="failed to get container status \"f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943\": rpc error: code = NotFound desc = could not find container \"f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943\": container with ID starting with f68b377ebb493eaa750576b9eb4fce37e329b603a5ebc6dededfac0810c85943 not found: ID does not exist" Jul 23 14:12:05.829902 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.829878 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx"] Jul 23 14:12:05.835351 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:05.835331 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-854fd7f2-8554c895dd-czpdx"] Jul 23 14:12:06.563211 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:12:06.563178 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab05b437-9f97-4e88-94ee-082082dcd845" path="/var/lib/kubelet/pods/ab05b437-9f97-4e88-94ee-082082dcd845/volumes" Jul 23 14:13:54.423931 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.423898 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg"] Jul 23 14:13:54.424386 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.424265 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab05b437-9f97-4e88-94ee-082082dcd845" containerName="maas-api" Jul 23 14:13:54.424386 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.424276 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab05b437-9f97-4e88-94ee-082082dcd845" containerName="maas-api" Jul 23 14:13:54.424386 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.424334 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab05b437-9f97-4e88-94ee-082082dcd845" containerName="maas-api" Jul 23 14:13:54.427256 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.427238 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.430736 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.430711 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:13:54.430913 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.430712 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ffcc91a2-serving-cert\"" Jul 23 14:13:54.430913 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.430808 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:13:54.431027 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.430930 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:13:54.443622 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.443600 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg"] Jul 23 14:13:54.532109 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.532067 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c72d96b8-be78-48a8-8f5a-c9301c234200-maas-api-tls\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.532109 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.532110 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tgbj\" (UniqueName: \"kubernetes.io/projected/c72d96b8-be78-48a8-8f5a-c9301c234200-kube-api-access-2tgbj\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.532348 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.532146 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c72d96b8-be78-48a8-8f5a-c9301c234200-openshift-service-ca\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.633261 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.633230 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c72d96b8-be78-48a8-8f5a-c9301c234200-maas-api-tls\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.633441 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.633271 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tgbj\" (UniqueName: \"kubernetes.io/projected/c72d96b8-be78-48a8-8f5a-c9301c234200-kube-api-access-2tgbj\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.633441 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.633305 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c72d96b8-be78-48a8-8f5a-c9301c234200-openshift-service-ca\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.633921 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.633904 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c72d96b8-be78-48a8-8f5a-c9301c234200-openshift-service-ca\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.635683 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.635657 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c72d96b8-be78-48a8-8f5a-c9301c234200-maas-api-tls\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.649168 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.649137 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tgbj\" (UniqueName: \"kubernetes.io/projected/c72d96b8-be78-48a8-8f5a-c9301c234200-kube-api-access-2tgbj\") pod \"maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.737786 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.737701 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:54.870985 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:54.870962 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg"] Jul 23 14:13:54.873668 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:13:54.873647 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72d96b8_be78_48a8_8f5a_c9301c234200.slice/crio-0b5fbae84030566ee178cd4a52030ab6966da909dffaacd138f0f14639afab3f WatchSource:0}: Error finding container 0b5fbae84030566ee178cd4a52030ab6966da909dffaacd138f0f14639afab3f: Status 404 returned error can't find the container with id 0b5fbae84030566ee178cd4a52030ab6966da909dffaacd138f0f14639afab3f Jul 23 14:13:55.278197 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:55.278160 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" event={"ID":"c72d96b8-be78-48a8-8f5a-c9301c234200","Type":"ContainerStarted","Data":"0b5fbae84030566ee178cd4a52030ab6966da909dffaacd138f0f14639afab3f"} Jul 23 14:13:57.289792 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:57.289753 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" event={"ID":"c72d96b8-be78-48a8-8f5a-c9301c234200","Type":"ContainerStarted","Data":"93ac93992ffa35d1e94cfd5e12b78bfa92a7b355fbd628ff769e9989db93ac48"} Jul 23 14:13:57.290219 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:57.289906 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:13:57.317330 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:13:57.317282 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" podStartSLOduration=1.607736582 podStartE2EDuration="3.317266203s" totalImagesPullingTime="1.709529621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:13:57.315664785 +0000 UTC m=+1989.337376665" watchObservedRunningTime="2026-07-23 14:13:57.317266203 +0000 UTC m=+1989.338978082" Jul 23 14:14:03.299521 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:03.299493 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:14:17.213928 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.213776 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg"] Jul 23 14:14:17.215324 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.214050 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" podUID="c72d96b8-be78-48a8-8f5a-c9301c234200" containerName="maas-api" containerID="cri-o://93ac93992ffa35d1e94cfd5e12b78bfa92a7b355fbd628ff769e9989db93ac48" gracePeriod=30 Jul 23 14:14:17.383041 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.383010 2582 generic.go:356] "Generic (PLEG): container finished" podID="c72d96b8-be78-48a8-8f5a-c9301c234200" containerID="93ac93992ffa35d1e94cfd5e12b78bfa92a7b355fbd628ff769e9989db93ac48" exitCode=0 Jul 23 14:14:17.383231 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.383086 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" event={"ID":"c72d96b8-be78-48a8-8f5a-c9301c234200","Type":"ContainerDied","Data":"93ac93992ffa35d1e94cfd5e12b78bfa92a7b355fbd628ff769e9989db93ac48"} Jul 23 14:14:17.458577 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.458554 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:14:17.540000 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.539904 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c72d96b8-be78-48a8-8f5a-c9301c234200-maas-api-tls\") pod \"c72d96b8-be78-48a8-8f5a-c9301c234200\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " Jul 23 14:14:17.540000 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.539951 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c72d96b8-be78-48a8-8f5a-c9301c234200-openshift-service-ca\") pod \"c72d96b8-be78-48a8-8f5a-c9301c234200\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " Jul 23 14:14:17.540000 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.539997 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tgbj\" (UniqueName: \"kubernetes.io/projected/c72d96b8-be78-48a8-8f5a-c9301c234200-kube-api-access-2tgbj\") pod \"c72d96b8-be78-48a8-8f5a-c9301c234200\" (UID: \"c72d96b8-be78-48a8-8f5a-c9301c234200\") " Jul 23 14:14:17.540358 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.540334 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c72d96b8-be78-48a8-8f5a-c9301c234200-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c72d96b8-be78-48a8-8f5a-c9301c234200" (UID: "c72d96b8-be78-48a8-8f5a-c9301c234200"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:14:17.542110 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.542074 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c72d96b8-be78-48a8-8f5a-c9301c234200-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c72d96b8-be78-48a8-8f5a-c9301c234200" (UID: "c72d96b8-be78-48a8-8f5a-c9301c234200"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:14:17.542256 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.542166 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c72d96b8-be78-48a8-8f5a-c9301c234200-kube-api-access-2tgbj" (OuterVolumeSpecName: "kube-api-access-2tgbj") pod "c72d96b8-be78-48a8-8f5a-c9301c234200" (UID: "c72d96b8-be78-48a8-8f5a-c9301c234200"). InnerVolumeSpecName "kube-api-access-2tgbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:14:17.641337 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.641299 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c72d96b8-be78-48a8-8f5a-c9301c234200-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:14:17.641337 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.641330 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tgbj\" (UniqueName: \"kubernetes.io/projected/c72d96b8-be78-48a8-8f5a-c9301c234200-kube-api-access-2tgbj\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:14:17.641337 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:17.641341 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c72d96b8-be78-48a8-8f5a-c9301c234200-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:14:18.389280 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:18.389256 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" Jul 23 14:14:18.389668 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:18.389286 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg" event={"ID":"c72d96b8-be78-48a8-8f5a-c9301c234200","Type":"ContainerDied","Data":"0b5fbae84030566ee178cd4a52030ab6966da909dffaacd138f0f14639afab3f"} Jul 23 14:14:18.389668 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:18.389343 2582 scope.go:117] "RemoveContainer" containerID="93ac93992ffa35d1e94cfd5e12b78bfa92a7b355fbd628ff769e9989db93ac48" Jul 23 14:14:18.413495 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:18.413468 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg"] Jul 23 14:14:18.418092 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:18.418060 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ffcc91a2-67d964b745-wf4rg"] Jul 23 14:14:18.569100 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:18.569066 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c72d96b8-be78-48a8-8f5a-c9301c234200" path="/var/lib/kubelet/pods/c72d96b8-be78-48a8-8f5a-c9301c234200/volumes" Jul 23 14:14:52.878778 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.878739 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2"] Jul 23 14:14:52.879305 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.879237 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c72d96b8-be78-48a8-8f5a-c9301c234200" containerName="maas-api" Jul 23 14:14:52.879305 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.879256 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c72d96b8-be78-48a8-8f5a-c9301c234200" containerName="maas-api" Jul 23 14:14:52.879425 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.879332 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c72d96b8-be78-48a8-8f5a-c9301c234200" containerName="maas-api" Jul 23 14:14:52.882341 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.882321 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:52.887413 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.887387 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:14:52.887530 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.887435 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-b34b8623-serving-cert\"" Jul 23 14:14:52.887530 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.887516 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:14:52.888465 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.888447 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:14:52.903933 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.903908 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2"] Jul 23 14:14:52.952532 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.952493 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clv4r\" (UniqueName: \"kubernetes.io/projected/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-kube-api-access-clv4r\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:52.952719 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.952591 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-maas-api-tls\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:52.952719 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:52.952665 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-openshift-service-ca\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.053281 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.053251 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clv4r\" (UniqueName: \"kubernetes.io/projected/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-kube-api-access-clv4r\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.053471 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.053306 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-maas-api-tls\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.053471 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.053357 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-openshift-service-ca\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.054034 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.054007 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-openshift-service-ca\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.055906 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.055883 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-maas-api-tls\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.063311 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.063290 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clv4r\" (UniqueName: \"kubernetes.io/projected/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-kube-api-access-clv4r\") pod \"maas-api-e2e-mt-b34b8623-599b77797d-2k9q2\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.192675 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.192592 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:53.334261 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.334203 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2"] Jul 23 14:14:53.337690 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:14:53.337646 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce2ebc7_c6e0_424c_bdf7_dbda57d6c288.slice/crio-b0ccb52761f911a72c520a1c2e9a84c20ba463e29744ddca188eb6cbe9c2dd61 WatchSource:0}: Error finding container b0ccb52761f911a72c520a1c2e9a84c20ba463e29744ddca188eb6cbe9c2dd61: Status 404 returned error can't find the container with id b0ccb52761f911a72c520a1c2e9a84c20ba463e29744ddca188eb6cbe9c2dd61 Jul 23 14:14:53.339157 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.339139 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 14:14:53.546318 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:53.546221 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" event={"ID":"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288","Type":"ContainerStarted","Data":"b0ccb52761f911a72c520a1c2e9a84c20ba463e29744ddca188eb6cbe9c2dd61"} Jul 23 14:14:54.552096 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:54.552056 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" event={"ID":"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288","Type":"ContainerStarted","Data":"c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79"} Jul 23 14:14:54.552580 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:54.552138 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:14:54.572227 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:14:54.572181 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" podStartSLOduration=2.188943767 podStartE2EDuration="2.57216646s" totalImagesPullingTime="383.222693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:14:54.570917041 +0000 UTC m=+2046.592628940" watchObservedRunningTime="2026-07-23 14:14:54.57216646 +0000 UTC m=+2046.593878338" Jul 23 14:15:00.563861 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:00.563823 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:15:48.518134 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:48.518099 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:15:48.520977 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:48.520957 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:15:48.531001 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:48.530984 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:15:48.533582 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:48.533565 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:15:56.442351 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.442320 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2"] Jul 23 14:15:56.442791 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.442539 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" podUID="7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" containerName="maas-api" containerID="cri-o://c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79" gracePeriod=30 Jul 23 14:15:56.692741 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.692685 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:15:56.814824 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.814790 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" containerID="c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79" exitCode=0 Jul 23 14:15:56.815049 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.814869 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" Jul 23 14:15:56.815049 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.814887 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" event={"ID":"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288","Type":"ContainerDied","Data":"c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79"} Jul 23 14:15:56.815049 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.814942 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2" event={"ID":"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288","Type":"ContainerDied","Data":"b0ccb52761f911a72c520a1c2e9a84c20ba463e29744ddca188eb6cbe9c2dd61"} Jul 23 14:15:56.815049 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.814969 2582 scope.go:117] "RemoveContainer" containerID="c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79" Jul 23 14:15:56.822782 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.822767 2582 scope.go:117] "RemoveContainer" containerID="c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79" Jul 23 14:15:56.823055 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:15:56.823038 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79\": container with ID starting with c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79 not found: ID does not exist" containerID="c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79" Jul 23 14:15:56.823111 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.823064 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79"} err="failed to get container status \"c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79\": rpc error: code = NotFound desc = could not find container \"c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79\": container with ID starting with c41393dd0251dfa55569f793aa4e29c375bedcef6479398b75cb4852d7ee1e79 not found: ID does not exist" Jul 23 14:15:56.844514 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.844488 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clv4r\" (UniqueName: \"kubernetes.io/projected/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-kube-api-access-clv4r\") pod \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " Jul 23 14:15:56.844662 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.844553 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-maas-api-tls\") pod \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " Jul 23 14:15:56.844662 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.844638 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-openshift-service-ca\") pod \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\" (UID: \"7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288\") " Jul 23 14:15:56.845029 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.845006 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" (UID: "7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:15:56.846563 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.846535 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-kube-api-access-clv4r" (OuterVolumeSpecName: "kube-api-access-clv4r") pod "7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" (UID: "7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288"). InnerVolumeSpecName "kube-api-access-clv4r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:15:56.846737 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.846714 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" (UID: "7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:15:56.945751 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.945671 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clv4r\" (UniqueName: \"kubernetes.io/projected/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-kube-api-access-clv4r\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:15:56.945751 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.945698 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:15:56.945751 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:56.945707 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:15:57.142176 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:57.142145 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2"] Jul 23 14:15:57.147495 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:57.147469 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-b34b8623-599b77797d-2k9q2"] Jul 23 14:15:58.563712 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:15:58.563677 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" path="/var/lib/kubelet/pods/7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288/volumes" Jul 23 14:18:07.077569 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.077532 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k"] Jul 23 14:18:07.078124 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.078087 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" containerName="maas-api" Jul 23 14:18:07.078124 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.078106 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" containerName="maas-api" Jul 23 14:18:07.078241 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.078216 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce2ebc7-c6e0-424c-bdf7-dbda57d6c288" containerName="maas-api" Jul 23 14:18:07.081479 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.081459 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.088453 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.088428 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:18:07.088639 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.088475 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-23f4ff-serving-cert\"" Jul 23 14:18:07.089503 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.089479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:18:07.089605 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.089479 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:18:07.094446 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.094426 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k"] Jul 23 14:18:07.194831 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.194799 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ad418f1-725d-4074-833d-b946fb9f3703-maas-api-tls\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.195023 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.194877 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad418f1-725d-4074-833d-b946fb9f3703-openshift-service-ca\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.195023 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.194932 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62dkf\" (UniqueName: \"kubernetes.io/projected/3ad418f1-725d-4074-833d-b946fb9f3703-kube-api-access-62dkf\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.295680 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.295650 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad418f1-725d-4074-833d-b946fb9f3703-openshift-service-ca\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.295899 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.295726 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62dkf\" (UniqueName: \"kubernetes.io/projected/3ad418f1-725d-4074-833d-b946fb9f3703-kube-api-access-62dkf\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.295899 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.295781 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ad418f1-725d-4074-833d-b946fb9f3703-maas-api-tls\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.296444 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.296424 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad418f1-725d-4074-833d-b946fb9f3703-openshift-service-ca\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.298275 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.298254 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ad418f1-725d-4074-833d-b946fb9f3703-maas-api-tls\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.310354 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.310328 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62dkf\" (UniqueName: \"kubernetes.io/projected/3ad418f1-725d-4074-833d-b946fb9f3703-kube-api-access-62dkf\") pod \"maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.392568 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.392536 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:07.523956 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:07.523876 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k"] Jul 23 14:18:07.526391 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:18:07.526354 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ad418f1_725d_4074_833d_b946fb9f3703.slice/crio-0695d1366889590a1cb2fad8d06c49c051cb473b686fdd2758a27123fbfd850b WatchSource:0}: Error finding container 0695d1366889590a1cb2fad8d06c49c051cb473b686fdd2758a27123fbfd850b: Status 404 returned error can't find the container with id 0695d1366889590a1cb2fad8d06c49c051cb473b686fdd2758a27123fbfd850b Jul 23 14:18:08.402921 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:08.402882 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" event={"ID":"3ad418f1-725d-4074-833d-b946fb9f3703","Type":"ContainerStarted","Data":"9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060"} Jul 23 14:18:08.403310 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:08.402939 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" event={"ID":"3ad418f1-725d-4074-833d-b946fb9f3703","Type":"ContainerStarted","Data":"0695d1366889590a1cb2fad8d06c49c051cb473b686fdd2758a27123fbfd850b"} Jul 23 14:18:08.403310 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:08.402994 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:08.426993 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:08.426947 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" podStartSLOduration=0.989297731 podStartE2EDuration="1.426929032s" totalImagesPullingTime="437.631301ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:18:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:18:08.425896833 +0000 UTC m=+2240.447608713" watchObservedRunningTime="2026-07-23 14:18:08.426929032 +0000 UTC m=+2240.448640911" Jul 23 14:18:14.413001 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:14.412931 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:18:15.161891 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.161853 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg"] Jul 23 14:18:15.165101 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.165082 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.168274 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.168252 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-80e3ed-serving-cert\"" Jul 23 14:18:15.180364 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.180340 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg"] Jul 23 14:18:15.265799 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.265770 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5598584e-1e7b-4aa6-892b-86c647690d23-maas-api-tls\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.266009 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.265812 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rltzt\" (UniqueName: \"kubernetes.io/projected/5598584e-1e7b-4aa6-892b-86c647690d23-kube-api-access-rltzt\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.266009 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.265890 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5598584e-1e7b-4aa6-892b-86c647690d23-openshift-service-ca\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.366333 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.366297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5598584e-1e7b-4aa6-892b-86c647690d23-maas-api-tls\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.366518 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.366344 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rltzt\" (UniqueName: \"kubernetes.io/projected/5598584e-1e7b-4aa6-892b-86c647690d23-kube-api-access-rltzt\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.366518 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.366372 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5598584e-1e7b-4aa6-892b-86c647690d23-openshift-service-ca\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.367027 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.367006 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5598584e-1e7b-4aa6-892b-86c647690d23-openshift-service-ca\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.368827 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.368795 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5598584e-1e7b-4aa6-892b-86c647690d23-maas-api-tls\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.377428 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.377407 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rltzt\" (UniqueName: \"kubernetes.io/projected/5598584e-1e7b-4aa6-892b-86c647690d23-kube-api-access-rltzt\") pod \"maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.476919 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.476792 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:15.606999 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:15.606975 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg"] Jul 23 14:18:15.609524 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:18:15.609482 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5598584e_1e7b_4aa6_892b_86c647690d23.slice/crio-03a3665497ff00c762a056db91ce3b3e3887323d2730f46e0e084db191e53ecd WatchSource:0}: Error finding container 03a3665497ff00c762a056db91ce3b3e3887323d2730f46e0e084db191e53ecd: Status 404 returned error can't find the container with id 03a3665497ff00c762a056db91ce3b3e3887323d2730f46e0e084db191e53ecd Jul 23 14:18:16.442013 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:16.441982 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" event={"ID":"5598584e-1e7b-4aa6-892b-86c647690d23","Type":"ContainerStarted","Data":"888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824"} Jul 23 14:18:16.442223 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:16.442039 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" event={"ID":"5598584e-1e7b-4aa6-892b-86c647690d23","Type":"ContainerStarted","Data":"03a3665497ff00c762a056db91ce3b3e3887323d2730f46e0e084db191e53ecd"} Jul 23 14:18:16.442223 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:16.442066 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:18:16.500895 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:16.500824 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" podStartSLOduration=1.09986835 podStartE2EDuration="1.50080054s" totalImagesPullingTime="400.93219ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:18:16.499514453 +0000 UTC m=+2248.521226351" watchObservedRunningTime="2026-07-23 14:18:16.50080054 +0000 UTC m=+2248.522512418" Jul 23 14:18:22.452442 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:18:22.452412 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:19:45.414640 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.414605 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs"] Jul 23 14:19:45.418252 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.418234 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:45.420984 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.420970 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pngt\"" Jul 23 14:19:45.432715 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.432684 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs"] Jul 23 14:19:45.571190 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.571154 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1baf8-087d-4cbb-b5d1-7e042c385f05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs\" (UID: \"33c1baf8-087d-4cbb-b5d1-7e042c385f05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:45.672722 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.672629 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1baf8-087d-4cbb-b5d1-7e042c385f05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs\" (UID: \"33c1baf8-087d-4cbb-b5d1-7e042c385f05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:45.673283 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.673261 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1baf8-087d-4cbb-b5d1-7e042c385f05-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs\" (UID: \"33c1baf8-087d-4cbb-b5d1-7e042c385f05\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:45.727457 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.727430 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:45.871471 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:45.871448 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs"] Jul 23 14:19:45.873768 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:19:45.873739 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c1baf8_087d_4cbb_b5d1_7e042c385f05.slice/crio-670e0bb11bdd6e3959c0cdaf0410214cd71c6f64c676c6938c3ac71491e5bffd WatchSource:0}: Error finding container 670e0bb11bdd6e3959c0cdaf0410214cd71c6f64c676c6938c3ac71491e5bffd: Status 404 returned error can't find the container with id 670e0bb11bdd6e3959c0cdaf0410214cd71c6f64c676c6938c3ac71491e5bffd Jul 23 14:19:46.848426 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:46.848391 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" event={"ID":"33c1baf8-087d-4cbb-b5d1-7e042c385f05","Type":"ContainerStarted","Data":"670e0bb11bdd6e3959c0cdaf0410214cd71c6f64c676c6938c3ac71491e5bffd"} Jul 23 14:19:48.859021 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:48.858987 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" event={"ID":"33c1baf8-087d-4cbb-b5d1-7e042c385f05","Type":"ContainerStarted","Data":"4bd51b3c98de722e974b34ec6f17497ef11bac74f44c62cd9ca0f83a12876809"} Jul 23 14:19:48.878468 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:48.878422 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" podStartSLOduration=1.433210775 podStartE2EDuration="3.878406705s" totalImagesPullingTime="2.44519593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:19:48.877670081 +0000 UTC m=+2340.899381978" watchObservedRunningTime="2026-07-23 14:19:48.878406705 +0000 UTC m=+2340.900118582" Jul 23 14:19:49.865274 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:49.865235 2582 generic.go:356] "Generic (PLEG): container finished" podID="33c1baf8-087d-4cbb-b5d1-7e042c385f05" containerID="4bd51b3c98de722e974b34ec6f17497ef11bac74f44c62cd9ca0f83a12876809" exitCode=0 Jul 23 14:19:49.865771 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:49.865328 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" event={"ID":"33c1baf8-087d-4cbb-b5d1-7e042c385f05","Type":"ContainerDied","Data":"4bd51b3c98de722e974b34ec6f17497ef11bac74f44c62cd9ca0f83a12876809"} Jul 23 14:19:50.992056 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:50.992028 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:51.122275 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:51.122196 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1baf8-087d-4cbb-b5d1-7e042c385f05-maas-api-service-ca\") pod \"33c1baf8-087d-4cbb-b5d1-7e042c385f05\" (UID: \"33c1baf8-087d-4cbb-b5d1-7e042c385f05\") " Jul 23 14:19:51.122542 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:51.122517 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33c1baf8-087d-4cbb-b5d1-7e042c385f05-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33c1baf8-087d-4cbb-b5d1-7e042c385f05" (UID: "33c1baf8-087d-4cbb-b5d1-7e042c385f05"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:19:51.223464 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:51.223413 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c1baf8-087d-4cbb-b5d1-7e042c385f05-maas-api-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:19:51.877881 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:51.877835 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" Jul 23 14:19:51.877881 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:51.877834 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs" event={"ID":"33c1baf8-087d-4cbb-b5d1-7e042c385f05","Type":"ContainerDied","Data":"670e0bb11bdd6e3959c0cdaf0410214cd71c6f64c676c6938c3ac71491e5bffd"} Jul 23 14:19:51.878093 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:51.877901 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="670e0bb11bdd6e3959c0cdaf0410214cd71c6f64c676c6938c3ac71491e5bffd" Jul 23 14:19:55.903608 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:55.903576 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k"] Jul 23 14:19:55.904019 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:55.903796 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" podUID="3ad418f1-725d-4074-833d-b946fb9f3703" containerName="maas-api" containerID="cri-o://9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060" gracePeriod=30 Jul 23 14:19:56.148378 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.148353 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:19:56.167035 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.166984 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ad418f1-725d-4074-833d-b946fb9f3703-maas-api-tls\") pod \"3ad418f1-725d-4074-833d-b946fb9f3703\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " Jul 23 14:19:56.167131 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.167048 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad418f1-725d-4074-833d-b946fb9f3703-openshift-service-ca\") pod \"3ad418f1-725d-4074-833d-b946fb9f3703\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " Jul 23 14:19:56.167177 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.167147 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62dkf\" (UniqueName: \"kubernetes.io/projected/3ad418f1-725d-4074-833d-b946fb9f3703-kube-api-access-62dkf\") pod \"3ad418f1-725d-4074-833d-b946fb9f3703\" (UID: \"3ad418f1-725d-4074-833d-b946fb9f3703\") " Jul 23 14:19:56.167361 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.167340 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ad418f1-725d-4074-833d-b946fb9f3703-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ad418f1-725d-4074-833d-b946fb9f3703" (UID: "3ad418f1-725d-4074-833d-b946fb9f3703"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:19:56.167441 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.167420 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ad418f1-725d-4074-833d-b946fb9f3703-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:19:56.169042 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.169013 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ad418f1-725d-4074-833d-b946fb9f3703-kube-api-access-62dkf" (OuterVolumeSpecName: "kube-api-access-62dkf") pod "3ad418f1-725d-4074-833d-b946fb9f3703" (UID: "3ad418f1-725d-4074-833d-b946fb9f3703"). InnerVolumeSpecName "kube-api-access-62dkf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:19:56.169152 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.169115 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ad418f1-725d-4074-833d-b946fb9f3703-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ad418f1-725d-4074-833d-b946fb9f3703" (UID: "3ad418f1-725d-4074-833d-b946fb9f3703"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:19:56.268202 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.268170 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ad418f1-725d-4074-833d-b946fb9f3703-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:19:56.268202 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.268196 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62dkf\" (UniqueName: \"kubernetes.io/projected/3ad418f1-725d-4074-833d-b946fb9f3703-kube-api-access-62dkf\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:19:56.904324 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.904291 2582 generic.go:356] "Generic (PLEG): container finished" podID="3ad418f1-725d-4074-833d-b946fb9f3703" containerID="9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060" exitCode=0 Jul 23 14:19:56.904776 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.904348 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" Jul 23 14:19:56.904776 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.904357 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" event={"ID":"3ad418f1-725d-4074-833d-b946fb9f3703","Type":"ContainerDied","Data":"9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060"} Jul 23 14:19:56.904776 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.904398 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k" event={"ID":"3ad418f1-725d-4074-833d-b946fb9f3703","Type":"ContainerDied","Data":"0695d1366889590a1cb2fad8d06c49c051cb473b686fdd2758a27123fbfd850b"} Jul 23 14:19:56.904776 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.904418 2582 scope.go:117] "RemoveContainer" containerID="9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060" Jul 23 14:19:56.912536 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.912519 2582 scope.go:117] "RemoveContainer" containerID="9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060" Jul 23 14:19:56.912755 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:19:56.912736 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060\": container with ID starting with 9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060 not found: ID does not exist" containerID="9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060" Jul 23 14:19:56.912809 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.912766 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060"} err="failed to get container status \"9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060\": rpc error: code = NotFound desc = could not find container \"9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060\": container with ID starting with 9e30e0346bca3a47580f004f75a4a9fbb5a7241e7df59a5f3444d8b6def52060 not found: ID does not exist" Jul 23 14:19:56.929198 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.929169 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k"] Jul 23 14:19:56.935131 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:56.935111 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-23f4ff-97d54578c-8zx6k"] Jul 23 14:19:58.562758 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:19:58.562727 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ad418f1-725d-4074-833d-b946fb9f3703" path="/var/lib/kubelet/pods/3ad418f1-725d-4074-833d-b946fb9f3703/volumes" Jul 23 14:20:00.541474 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:00.541439 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs"] Jul 23 14:20:00.546640 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:00.546606 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-23f4ff-6cwfs"] Jul 23 14:20:00.563117 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:00.563089 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c1baf8-087d-4cbb-b5d1-7e042c385f05" path="/var/lib/kubelet/pods/33c1baf8-087d-4cbb-b5d1-7e042c385f05/volumes" Jul 23 14:20:08.062214 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062175 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m"] Jul 23 14:20:08.062608 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062536 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c1baf8-087d-4cbb-b5d1-7e042c385f05" containerName="revoke-keys" Jul 23 14:20:08.062608 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062547 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c1baf8-087d-4cbb-b5d1-7e042c385f05" containerName="revoke-keys" Jul 23 14:20:08.062608 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062566 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ad418f1-725d-4074-833d-b946fb9f3703" containerName="maas-api" Jul 23 14:20:08.062608 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062572 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ad418f1-725d-4074-833d-b946fb9f3703" containerName="maas-api" Jul 23 14:20:08.062727 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062629 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ad418f1-725d-4074-833d-b946fb9f3703" containerName="maas-api" Jul 23 14:20:08.062727 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.062638 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c1baf8-087d-4cbb-b5d1-7e042c385f05" containerName="revoke-keys" Jul 23 14:20:08.066604 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.066587 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:08.069393 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.069370 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pngt\"" Jul 23 14:20:08.074125 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.073908 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m"] Jul 23 14:20:08.165297 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.165265 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m\" (UID: \"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:08.265759 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.265723 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m\" (UID: \"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:08.266342 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.266324 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m\" (UID: \"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:08.378043 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.378006 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:08.517973 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.517946 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m"] Jul 23 14:20:08.519855 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:20:08.519805 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb37dcdc1_2dc4_47a5_b43f_49d6ddd43453.slice/crio-a7a18a57fb4e3ee3a3a7a20253819a852f9c2b94c071bd26f95fa7c58f515fe6 WatchSource:0}: Error finding container a7a18a57fb4e3ee3a3a7a20253819a852f9c2b94c071bd26f95fa7c58f515fe6: Status 404 returned error can't find the container with id a7a18a57fb4e3ee3a3a7a20253819a852f9c2b94c071bd26f95fa7c58f515fe6 Jul 23 14:20:08.960084 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.960053 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" event={"ID":"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453","Type":"ContainerStarted","Data":"375a75025abf43b689a5153d5dbc6926268f736b7c56630405ade9cde113ab2b"} Jul 23 14:20:08.960240 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.960106 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" event={"ID":"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453","Type":"ContainerStarted","Data":"a7a18a57fb4e3ee3a3a7a20253819a852f9c2b94c071bd26f95fa7c58f515fe6"} Jul 23 14:20:08.977769 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:08.977725 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" podStartSLOduration=0.977710809 podStartE2EDuration="977.710809ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:20:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:20:08.976487755 +0000 UTC m=+2360.998199637" watchObservedRunningTime="2026-07-23 14:20:08.977710809 +0000 UTC m=+2360.999422687" Jul 23 14:20:09.966096 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:09.966048 2582 generic.go:356] "Generic (PLEG): container finished" podID="b37dcdc1-2dc4-47a5-b43f-49d6ddd43453" containerID="375a75025abf43b689a5153d5dbc6926268f736b7c56630405ade9cde113ab2b" exitCode=0 Jul 23 14:20:09.966096 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:09.966099 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" event={"ID":"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453","Type":"ContainerDied","Data":"375a75025abf43b689a5153d5dbc6926268f736b7c56630405ade9cde113ab2b"} Jul 23 14:20:11.106462 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.106432 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:11.189112 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.189084 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453-maas-api-service-ca\") pod \"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453\" (UID: \"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453\") " Jul 23 14:20:11.189479 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.189458 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b37dcdc1-2dc4-47a5-b43f-49d6ddd43453" (UID: "b37dcdc1-2dc4-47a5-b43f-49d6ddd43453"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:20:11.290891 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.290784 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453-maas-api-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:20:11.975530 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.975495 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" event={"ID":"b37dcdc1-2dc4-47a5-b43f-49d6ddd43453","Type":"ContainerDied","Data":"a7a18a57fb4e3ee3a3a7a20253819a852f9c2b94c071bd26f95fa7c58f515fe6"} Jul 23 14:20:11.975673 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.975550 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a7a18a57fb4e3ee3a3a7a20253819a852f9c2b94c071bd26f95fa7c58f515fe6" Jul 23 14:20:11.975673 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:11.975510 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m" Jul 23 14:20:18.613510 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.613479 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg"] Jul 23 14:20:18.613945 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.613717 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" podUID="5598584e-1e7b-4aa6-892b-86c647690d23" containerName="maas-api" containerID="cri-o://888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824" gracePeriod=30 Jul 23 14:20:18.855012 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.854985 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:20:18.962515 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.962424 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rltzt\" (UniqueName: \"kubernetes.io/projected/5598584e-1e7b-4aa6-892b-86c647690d23-kube-api-access-rltzt\") pod \"5598584e-1e7b-4aa6-892b-86c647690d23\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " Jul 23 14:20:18.962515 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.962474 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5598584e-1e7b-4aa6-892b-86c647690d23-maas-api-tls\") pod \"5598584e-1e7b-4aa6-892b-86c647690d23\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " Jul 23 14:20:18.962769 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.962529 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5598584e-1e7b-4aa6-892b-86c647690d23-openshift-service-ca\") pod \"5598584e-1e7b-4aa6-892b-86c647690d23\" (UID: \"5598584e-1e7b-4aa6-892b-86c647690d23\") " Jul 23 14:20:18.962975 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.962943 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5598584e-1e7b-4aa6-892b-86c647690d23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5598584e-1e7b-4aa6-892b-86c647690d23" (UID: "5598584e-1e7b-4aa6-892b-86c647690d23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:20:18.964575 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.964550 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5598584e-1e7b-4aa6-892b-86c647690d23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5598584e-1e7b-4aa6-892b-86c647690d23" (UID: "5598584e-1e7b-4aa6-892b-86c647690d23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:20:18.964677 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:18.964622 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5598584e-1e7b-4aa6-892b-86c647690d23-kube-api-access-rltzt" (OuterVolumeSpecName: "kube-api-access-rltzt") pod "5598584e-1e7b-4aa6-892b-86c647690d23" (UID: "5598584e-1e7b-4aa6-892b-86c647690d23"). InnerVolumeSpecName "kube-api-access-rltzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:20:19.008523 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.008492 2582 generic.go:356] "Generic (PLEG): container finished" podID="5598584e-1e7b-4aa6-892b-86c647690d23" containerID="888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824" exitCode=0 Jul 23 14:20:19.008686 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.008543 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" event={"ID":"5598584e-1e7b-4aa6-892b-86c647690d23","Type":"ContainerDied","Data":"888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824"} Jul 23 14:20:19.008686 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.008557 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" Jul 23 14:20:19.008686 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.008626 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg" event={"ID":"5598584e-1e7b-4aa6-892b-86c647690d23","Type":"ContainerDied","Data":"03a3665497ff00c762a056db91ce3b3e3887323d2730f46e0e084db191e53ecd"} Jul 23 14:20:19.008686 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.008657 2582 scope.go:117] "RemoveContainer" containerID="888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824" Jul 23 14:20:19.016958 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.016940 2582 scope.go:117] "RemoveContainer" containerID="888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824" Jul 23 14:20:19.017194 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:20:19.017176 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824\": container with ID starting with 888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824 not found: ID does not exist" containerID="888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824" Jul 23 14:20:19.017256 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.017203 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824"} err="failed to get container status \"888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824\": rpc error: code = NotFound desc = could not find container \"888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824\": container with ID starting with 888746a43b949676f15324f116d8492f43febf379aae8cb7ec3c665cc15a3824 not found: ID does not exist" Jul 23 14:20:19.033891 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.033869 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg"] Jul 23 14:20:19.036937 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.036914 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-80e3ed-558d479fc6-llfxg"] Jul 23 14:20:19.063440 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.063419 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rltzt\" (UniqueName: \"kubernetes.io/projected/5598584e-1e7b-4aa6-892b-86c647690d23-kube-api-access-rltzt\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:20:19.063440 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.063440 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5598584e-1e7b-4aa6-892b-86c647690d23-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:20:19.063545 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:19.063452 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5598584e-1e7b-4aa6-892b-86c647690d23-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:20:20.563939 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:20.563910 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5598584e-1e7b-4aa6-892b-86c647690d23" path="/var/lib/kubelet/pods/5598584e-1e7b-4aa6-892b-86c647690d23/volumes" Jul 23 14:20:23.136403 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:23.136374 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m"] Jul 23 14:20:23.138657 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:23.138635 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-80e3ed-42l2m"] Jul 23 14:20:24.563532 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:24.563499 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b37dcdc1-2dc4-47a5-b43f-49d6ddd43453" path="/var/lib/kubelet/pods/b37dcdc1-2dc4-47a5-b43f-49d6ddd43453/volumes" Jul 23 14:20:33.898384 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898353 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z"] Jul 23 14:20:33.898740 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898673 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b37dcdc1-2dc4-47a5-b43f-49d6ddd43453" containerName="revoke-keys" Jul 23 14:20:33.898740 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898684 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b37dcdc1-2dc4-47a5-b43f-49d6ddd43453" containerName="revoke-keys" Jul 23 14:20:33.898740 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898703 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5598584e-1e7b-4aa6-892b-86c647690d23" containerName="maas-api" Jul 23 14:20:33.898740 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898709 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5598584e-1e7b-4aa6-892b-86c647690d23" containerName="maas-api" Jul 23 14:20:33.898902 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898757 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5598584e-1e7b-4aa6-892b-86c647690d23" containerName="maas-api" Jul 23 14:20:33.898902 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.898767 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b37dcdc1-2dc4-47a5-b43f-49d6ddd43453" containerName="revoke-keys" Jul 23 14:20:33.901865 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.901826 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:33.906280 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.906262 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:20:33.906468 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.906455 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e8799a-serving-cert\"" Jul 23 14:20:33.906593 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.906575 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:20:33.906754 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.906741 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:20:33.913037 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.913015 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z"] Jul 23 14:20:33.995027 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.994985 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2v95\" (UniqueName: \"kubernetes.io/projected/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-kube-api-access-t2v95\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:33.995027 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.995036 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-maas-api-tls\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:33.995247 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:33.995055 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.096198 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.096160 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-maas-api-tls\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.096198 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.096197 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.096457 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.096316 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2v95\" (UniqueName: \"kubernetes.io/projected/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-kube-api-access-t2v95\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.096959 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.096937 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-openshift-service-ca\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.098523 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.098504 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-maas-api-tls\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.108112 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.108091 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2v95\" (UniqueName: \"kubernetes.io/projected/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-kube-api-access-t2v95\") pod \"maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.212053 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.211974 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:34.342136 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.342108 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z"] Jul 23 14:20:34.344042 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:20:34.344018 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7cf43c_ee14_4475_9ae2_3cf7346c1983.slice/crio-7886eea7b700ca38d09274bcd2e630191831ff7ba6cfebed4542fe9e50460f6a WatchSource:0}: Error finding container 7886eea7b700ca38d09274bcd2e630191831ff7ba6cfebed4542fe9e50460f6a: Status 404 returned error can't find the container with id 7886eea7b700ca38d09274bcd2e630191831ff7ba6cfebed4542fe9e50460f6a Jul 23 14:20:34.345646 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:34.345626 2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 14:20:35.084134 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:35.084100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" event={"ID":"8b7cf43c-ee14-4475-9ae2-3cf7346c1983","Type":"ContainerStarted","Data":"7886eea7b700ca38d09274bcd2e630191831ff7ba6cfebed4542fe9e50460f6a"} Jul 23 14:20:36.090364 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:36.090330 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" event={"ID":"8b7cf43c-ee14-4475-9ae2-3cf7346c1983","Type":"ContainerStarted","Data":"e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7"} Jul 23 14:20:36.090817 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:36.090457 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:36.111861 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:36.111765 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" podStartSLOduration=2.359083746 podStartE2EDuration="3.111742748s" totalImagesPullingTime="752.659002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:20:36.111166146 +0000 UTC m=+2388.132878025" watchObservedRunningTime="2026-07-23 14:20:36.111742748 +0000 UTC m=+2388.133454628" Jul 23 14:20:42.100997 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.100971 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:20:42.207590 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.207562 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx"] Jul 23 14:20:42.211263 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.211240 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.214358 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.214337 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-636249-serving-cert\"" Jul 23 14:20:42.237291 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.237261 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx"] Jul 23 14:20:42.277286 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.277258 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.277458 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.277292 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq5qq\" (UniqueName: \"kubernetes.io/projected/d8aa8acd-ca64-4f25-949c-01ea93e496e9-kube-api-access-fq5qq\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.277458 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.277411 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8aa8acd-ca64-4f25-949c-01ea93e496e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.378384 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.378347 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8aa8acd-ca64-4f25-949c-01ea93e496e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.378547 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.378423 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.378547 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.378447 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5qq\" (UniqueName: \"kubernetes.io/projected/d8aa8acd-ca64-4f25-949c-01ea93e496e9-kube-api-access-fq5qq\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.378622 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:20:42.378562 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-636249-serving-cert: secret "maas-api-e2e-api-b-636249-serving-cert" not found Jul 23 14:20:42.378656 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:20:42.378627 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls podName:d8aa8acd-ca64-4f25-949c-01ea93e496e9 nodeName:}" failed. No retries permitted until 2026-07-23 14:20:42.878611534 +0000 UTC m=+2394.900323395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls") pod "maas-api-e2e-api-b-636249-748dc49f68-656sx" (UID: "d8aa8acd-ca64-4f25-949c-01ea93e496e9") : secret "maas-api-e2e-api-b-636249-serving-cert" not found Jul 23 14:20:42.379019 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.379003 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8aa8acd-ca64-4f25-949c-01ea93e496e9-openshift-service-ca\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.388079 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.388055 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq5qq\" (UniqueName: \"kubernetes.io/projected/d8aa8acd-ca64-4f25-949c-01ea93e496e9-kube-api-access-fq5qq\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.881500 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.881465 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:42.884059 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:42.884034 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls\") pod \"maas-api-e2e-api-b-636249-748dc49f68-656sx\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:43.121750 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:43.121713 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:43.270146 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:43.270114 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx"] Jul 23 14:20:44.129016 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:44.128969 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" event={"ID":"d8aa8acd-ca64-4f25-949c-01ea93e496e9","Type":"ContainerStarted","Data":"1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7"} Jul 23 14:20:44.129470 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:44.129043 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" event={"ID":"d8aa8acd-ca64-4f25-949c-01ea93e496e9","Type":"ContainerStarted","Data":"774001d8506e05c4a5d364e83eef304df640e2697b301a3d1015af6a418f1303"} Jul 23 14:20:44.129470 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:44.129113 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:44.162410 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:44.162289 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" podStartSLOduration=1.5373995040000001 podStartE2EDuration="2.162272155s" totalImagesPullingTime="624.872651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:20:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:20:44.161653169 +0000 UTC m=+2396.183365070" watchObservedRunningTime="2026-07-23 14:20:44.162272155 +0000 UTC m=+2396.183984050" Jul 23 14:20:48.554145 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:48.554108 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:20:48.556625 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:48.556605 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:20:48.569090 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:48.569065 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:20:48.572191 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:48.572170 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:20:50.139554 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:50.139525 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:20:54.521193 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.521152 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b"] Jul 23 14:20:54.524060 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.524042 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:54.526779 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.526757 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pngt\"" Jul 23 14:20:54.532286 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.532261 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b"] Jul 23 14:20:54.581730 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.581692 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/555e18a5-56a4-4116-b62c-734e2c0c2d5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8799a-skr8b\" (UID: \"555e18a5-56a4-4116-b62c-734e2c0c2d5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:54.683051 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.683007 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/555e18a5-56a4-4116-b62c-734e2c0c2d5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8799a-skr8b\" (UID: \"555e18a5-56a4-4116-b62c-734e2c0c2d5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:54.683686 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.683661 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/555e18a5-56a4-4116-b62c-734e2c0c2d5e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-e8799a-skr8b\" (UID: \"555e18a5-56a4-4116-b62c-734e2c0c2d5e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:54.836176 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.836081 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:54.969137 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:54.969115 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b"] Jul 23 14:20:54.971652 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:20:54.971617 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod555e18a5_56a4_4116_b62c_734e2c0c2d5e.slice/crio-c83cd3f9d3f8ef56171982505d4c218021e98461f69abc86196eb7afea119b42 WatchSource:0}: Error finding container c83cd3f9d3f8ef56171982505d4c218021e98461f69abc86196eb7afea119b42: Status 404 returned error can't find the container with id c83cd3f9d3f8ef56171982505d4c218021e98461f69abc86196eb7afea119b42 Jul 23 14:20:55.185881 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:55.185826 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" event={"ID":"555e18a5-56a4-4116-b62c-734e2c0c2d5e","Type":"ContainerStarted","Data":"1c903bc8f1439682d18824db80906e7228b420725abb077c22a9e7aa72e11e01"} Jul 23 14:20:55.186054 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:55.185900 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" event={"ID":"555e18a5-56a4-4116-b62c-734e2c0c2d5e","Type":"ContainerStarted","Data":"c83cd3f9d3f8ef56171982505d4c218021e98461f69abc86196eb7afea119b42"} Jul 23 14:20:55.207350 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:55.207296 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" podStartSLOduration=1.207280056 podStartE2EDuration="1.207280056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:20:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:20:55.204985357 +0000 UTC m=+2407.226697248" watchObservedRunningTime="2026-07-23 14:20:55.207280056 +0000 UTC m=+2407.228991999" Jul 23 14:20:56.191727 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:56.191690 2582 generic.go:356] "Generic (PLEG): container finished" podID="555e18a5-56a4-4116-b62c-734e2c0c2d5e" containerID="1c903bc8f1439682d18824db80906e7228b420725abb077c22a9e7aa72e11e01" exitCode=0 Jul 23 14:20:56.192129 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:56.191769 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" event={"ID":"555e18a5-56a4-4116-b62c-734e2c0c2d5e","Type":"ContainerDied","Data":"1c903bc8f1439682d18824db80906e7228b420725abb077c22a9e7aa72e11e01"} Jul 23 14:20:57.347900 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:57.347867 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:57.414536 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:57.414505 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/555e18a5-56a4-4116-b62c-734e2c0c2d5e-maas-api-service-ca\") pod \"555e18a5-56a4-4116-b62c-734e2c0c2d5e\" (UID: \"555e18a5-56a4-4116-b62c-734e2c0c2d5e\") " Jul 23 14:20:57.414871 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:57.414824 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/555e18a5-56a4-4116-b62c-734e2c0c2d5e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "555e18a5-56a4-4116-b62c-734e2c0c2d5e" (UID: "555e18a5-56a4-4116-b62c-734e2c0c2d5e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:20:57.515894 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:57.515782 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/555e18a5-56a4-4116-b62c-734e2c0c2d5e-maas-api-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:20:58.204106 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:58.204077 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" Jul 23 14:20:58.204106 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:58.204100 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b" event={"ID":"555e18a5-56a4-4116-b62c-734e2c0c2d5e","Type":"ContainerDied","Data":"c83cd3f9d3f8ef56171982505d4c218021e98461f69abc86196eb7afea119b42"} Jul 23 14:20:58.204304 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:20:58.204141 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c83cd3f9d3f8ef56171982505d4c218021e98461f69abc86196eb7afea119b42" Jul 23 14:21:10.494781 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.494706 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z"] Jul 23 14:21:10.495144 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.494929 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" podUID="8b7cf43c-ee14-4475-9ae2-3cf7346c1983" containerName="maas-api" containerID="cri-o://e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7" gracePeriod=30 Jul 23 14:21:10.742129 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.742099 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:21:10.839906 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.839802 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2v95\" (UniqueName: \"kubernetes.io/projected/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-kube-api-access-t2v95\") pod \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " Jul 23 14:21:10.839906 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.839884 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-openshift-service-ca\") pod \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " Jul 23 14:21:10.840088 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.839928 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-maas-api-tls\") pod \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\" (UID: \"8b7cf43c-ee14-4475-9ae2-3cf7346c1983\") " Jul 23 14:21:10.840274 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.840248 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b7cf43c-ee14-4475-9ae2-3cf7346c1983" (UID: "8b7cf43c-ee14-4475-9ae2-3cf7346c1983"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:21:10.842024 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.841989 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b7cf43c-ee14-4475-9ae2-3cf7346c1983" (UID: "8b7cf43c-ee14-4475-9ae2-3cf7346c1983"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:21:10.842136 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.842035 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-kube-api-access-t2v95" (OuterVolumeSpecName: "kube-api-access-t2v95") pod "8b7cf43c-ee14-4475-9ae2-3cf7346c1983" (UID: "8b7cf43c-ee14-4475-9ae2-3cf7346c1983"). InnerVolumeSpecName "kube-api-access-t2v95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:21:10.940928 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.940893 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2v95\" (UniqueName: \"kubernetes.io/projected/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-kube-api-access-t2v95\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:10.940928 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.940918 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:10.940928 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:10.940929 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b7cf43c-ee14-4475-9ae2-3cf7346c1983-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:11.265591 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.265556 2582 generic.go:356] "Generic (PLEG): container finished" podID="8b7cf43c-ee14-4475-9ae2-3cf7346c1983" containerID="e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7" exitCode=0 Jul 23 14:21:11.265794 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.265605 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" Jul 23 14:21:11.265794 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.265637 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" event={"ID":"8b7cf43c-ee14-4475-9ae2-3cf7346c1983","Type":"ContainerDied","Data":"e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7"} Jul 23 14:21:11.265794 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.265688 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z" event={"ID":"8b7cf43c-ee14-4475-9ae2-3cf7346c1983","Type":"ContainerDied","Data":"7886eea7b700ca38d09274bcd2e630191831ff7ba6cfebed4542fe9e50460f6a"} Jul 23 14:21:11.265794 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.265710 2582 scope.go:117] "RemoveContainer" containerID="e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7" Jul 23 14:21:11.275002 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.274986 2582 scope.go:117] "RemoveContainer" containerID="e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7" Jul 23 14:21:11.275306 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:21:11.275289 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7\": container with ID starting with e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7 not found: ID does not exist" containerID="e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7" Jul 23 14:21:11.275364 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.275315 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7"} err="failed to get container status \"e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7\": rpc error: code = NotFound desc = could not find container \"e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7\": container with ID starting with e593227d58d219f930c69fbc3b2c3b2581cfc6052400edbc66a90ec5206e69e7 not found: ID does not exist" Jul 23 14:21:11.289596 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.289569 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z"] Jul 23 14:21:11.295470 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:11.295451 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e8799a-66d5cfbb9b-2cq2z"] Jul 23 14:21:12.563057 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:12.563023 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7cf43c-ee14-4475-9ae2-3cf7346c1983" path="/var/lib/kubelet/pods/8b7cf43c-ee14-4475-9ae2-3cf7346c1983/volumes" Jul 23 14:21:14.611711 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:14.611681 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b"] Jul 23 14:21:14.615184 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:14.615162 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-e8799a-skr8b"] Jul 23 14:21:16.563387 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:16.563352 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="555e18a5-56a4-4116-b62c-734e2c0c2d5e" path="/var/lib/kubelet/pods/555e18a5-56a4-4116-b62c-734e2c0c2d5e/volumes" Jul 23 14:21:21.843143 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843105 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4"] Jul 23 14:21:21.843635 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843616 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="555e18a5-56a4-4116-b62c-734e2c0c2d5e" containerName="revoke-keys" Jul 23 14:21:21.843705 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843639 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="555e18a5-56a4-4116-b62c-734e2c0c2d5e" containerName="revoke-keys" Jul 23 14:21:21.843705 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843678 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7cf43c-ee14-4475-9ae2-3cf7346c1983" containerName="maas-api" Jul 23 14:21:21.843705 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843687 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7cf43c-ee14-4475-9ae2-3cf7346c1983" containerName="maas-api" Jul 23 14:21:21.843874 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843763 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7cf43c-ee14-4475-9ae2-3cf7346c1983" containerName="maas-api" Jul 23 14:21:21.843874 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.843779 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="555e18a5-56a4-4116-b62c-734e2c0c2d5e" containerName="revoke-keys" Jul 23 14:21:21.848096 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.848075 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:21.850610 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.850593 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pngt\"" Jul 23 14:21:21.856753 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:21.856730 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4"] Jul 23 14:21:22.040905 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:22.040864 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef95338e-ff50-495b-b7d2-d4b6253f2e27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-636249-spnn4\" (UID: \"ef95338e-ff50-495b-b7d2-d4b6253f2e27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:22.142255 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:22.142217 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef95338e-ff50-495b-b7d2-d4b6253f2e27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-636249-spnn4\" (UID: \"ef95338e-ff50-495b-b7d2-d4b6253f2e27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:22.142794 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:22.142772 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef95338e-ff50-495b-b7d2-d4b6253f2e27-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-636249-spnn4\" (UID: \"ef95338e-ff50-495b-b7d2-d4b6253f2e27\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:22.159595 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:22.159562 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:22.495215 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:22.495022 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4"] Jul 23 14:21:22.497769 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:21:22.497744 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef95338e_ff50_495b_b7d2_d4b6253f2e27.slice/crio-284748e6941b29f90d2f2c721d5e5babbb60eefba3dadf1a6f5f7967852d6d9b WatchSource:0}: Error finding container 284748e6941b29f90d2f2c721d5e5babbb60eefba3dadf1a6f5f7967852d6d9b: Status 404 returned error can't find the container with id 284748e6941b29f90d2f2c721d5e5babbb60eefba3dadf1a6f5f7967852d6d9b Jul 23 14:21:23.324725 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:23.324685 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" event={"ID":"ef95338e-ff50-495b-b7d2-d4b6253f2e27","Type":"ContainerStarted","Data":"5f0d78b5113bb4cc4112cadf28b8f73c302a680b6ff86a24cf11a9c5c9a63643"} Jul 23 14:21:23.325114 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:23.324743 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" event={"ID":"ef95338e-ff50-495b-b7d2-d4b6253f2e27","Type":"ContainerStarted","Data":"284748e6941b29f90d2f2c721d5e5babbb60eefba3dadf1a6f5f7967852d6d9b"} Jul 23 14:21:23.341738 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:23.341694 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" podStartSLOduration=2.34167877 podStartE2EDuration="2.34167877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:21:23.340086043 +0000 UTC m=+2435.361797940" watchObservedRunningTime="2026-07-23 14:21:23.34167877 +0000 UTC m=+2435.363390648" Jul 23 14:21:24.330592 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:24.330550 2582 generic.go:356] "Generic (PLEG): container finished" podID="ef95338e-ff50-495b-b7d2-d4b6253f2e27" containerID="5f0d78b5113bb4cc4112cadf28b8f73c302a680b6ff86a24cf11a9c5c9a63643" exitCode=0 Jul 23 14:21:24.331047 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:24.330642 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" event={"ID":"ef95338e-ff50-495b-b7d2-d4b6253f2e27","Type":"ContainerDied","Data":"5f0d78b5113bb4cc4112cadf28b8f73c302a680b6ff86a24cf11a9c5c9a63643"} Jul 23 14:21:25.467243 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:25.467216 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:25.572159 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:25.572129 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef95338e-ff50-495b-b7d2-d4b6253f2e27-maas-api-service-ca\") pod \"ef95338e-ff50-495b-b7d2-d4b6253f2e27\" (UID: \"ef95338e-ff50-495b-b7d2-d4b6253f2e27\") " Jul 23 14:21:25.572472 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:25.572451 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef95338e-ff50-495b-b7d2-d4b6253f2e27-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ef95338e-ff50-495b-b7d2-d4b6253f2e27" (UID: "ef95338e-ff50-495b-b7d2-d4b6253f2e27"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:21:25.673503 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:25.673470 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef95338e-ff50-495b-b7d2-d4b6253f2e27-maas-api-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:26.342067 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:26.342038 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" Jul 23 14:21:26.342279 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:26.342033 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4" event={"ID":"ef95338e-ff50-495b-b7d2-d4b6253f2e27","Type":"ContainerDied","Data":"284748e6941b29f90d2f2c721d5e5babbb60eefba3dadf1a6f5f7967852d6d9b"} Jul 23 14:21:26.342279 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:26.342154 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="284748e6941b29f90d2f2c721d5e5babbb60eefba3dadf1a6f5f7967852d6d9b" Jul 23 14:21:37.379787 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.379752 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx"] Jul 23 14:21:37.380178 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.379973 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" podUID="d8aa8acd-ca64-4f25-949c-01ea93e496e9" containerName="maas-api" containerID="cri-o://1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7" gracePeriod=30 Jul 23 14:21:37.619823 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.619802 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:21:37.679363 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.679294 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8aa8acd-ca64-4f25-949c-01ea93e496e9-openshift-service-ca\") pod \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " Jul 23 14:21:37.679363 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.679354 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls\") pod \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " Jul 23 14:21:37.679523 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.679382 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fq5qq\" (UniqueName: \"kubernetes.io/projected/d8aa8acd-ca64-4f25-949c-01ea93e496e9-kube-api-access-fq5qq\") pod \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\" (UID: \"d8aa8acd-ca64-4f25-949c-01ea93e496e9\") " Jul 23 14:21:37.679715 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.679688 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8aa8acd-ca64-4f25-949c-01ea93e496e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d8aa8acd-ca64-4f25-949c-01ea93e496e9" (UID: "d8aa8acd-ca64-4f25-949c-01ea93e496e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:21:37.681407 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.681385 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d8aa8acd-ca64-4f25-949c-01ea93e496e9" (UID: "d8aa8acd-ca64-4f25-949c-01ea93e496e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:21:37.681510 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.681482 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8aa8acd-ca64-4f25-949c-01ea93e496e9-kube-api-access-fq5qq" (OuterVolumeSpecName: "kube-api-access-fq5qq") pod "d8aa8acd-ca64-4f25-949c-01ea93e496e9" (UID: "d8aa8acd-ca64-4f25-949c-01ea93e496e9"). InnerVolumeSpecName "kube-api-access-fq5qq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:21:37.780182 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.780154 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8aa8acd-ca64-4f25-949c-01ea93e496e9-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:37.780182 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.780179 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d8aa8acd-ca64-4f25-949c-01ea93e496e9-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:37.780182 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:37.780189 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fq5qq\" (UniqueName: \"kubernetes.io/projected/d8aa8acd-ca64-4f25-949c-01ea93e496e9-kube-api-access-fq5qq\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:21:38.403254 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.403220 2582 generic.go:356] "Generic (PLEG): container finished" podID="d8aa8acd-ca64-4f25-949c-01ea93e496e9" containerID="1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7" exitCode=0 Jul 23 14:21:38.403703 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.403272 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" Jul 23 14:21:38.403703 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.403279 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" event={"ID":"d8aa8acd-ca64-4f25-949c-01ea93e496e9","Type":"ContainerDied","Data":"1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7"} Jul 23 14:21:38.403703 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.403319 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx" event={"ID":"d8aa8acd-ca64-4f25-949c-01ea93e496e9","Type":"ContainerDied","Data":"774001d8506e05c4a5d364e83eef304df640e2697b301a3d1015af6a418f1303"} Jul 23 14:21:38.403703 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.403340 2582 scope.go:117] "RemoveContainer" containerID="1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7" Jul 23 14:21:38.413472 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.413449 2582 scope.go:117] "RemoveContainer" containerID="1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7" Jul 23 14:21:38.413761 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:21:38.413741 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7\": container with ID starting with 1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7 not found: ID does not exist" containerID="1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7" Jul 23 14:21:38.413865 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.413769 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7"} err="failed to get container status \"1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7\": rpc error: code = NotFound desc = could not find container \"1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7\": container with ID starting with 1866bc9fad43c6acf4507ef20032159c2d41cafe44c83ff5b4016a60cb9bb4a7 not found: ID does not exist" Jul 23 14:21:38.426188 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.426166 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx"] Jul 23 14:21:38.431741 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.431719 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-636249-748dc49f68-656sx"] Jul 23 14:21:38.563313 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:38.563273 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8aa8acd-ca64-4f25-949c-01ea93e496e9" path="/var/lib/kubelet/pods/d8aa8acd-ca64-4f25-949c-01ea93e496e9/volumes" Jul 23 14:21:41.936335 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:41.936303 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4"] Jul 23 14:21:41.939980 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:41.939953 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-636249-spnn4"] Jul 23 14:21:42.563023 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:42.562988 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef95338e-ff50-495b-b7d2-d4b6253f2e27" path="/var/lib/kubelet/pods/ef95338e-ff50-495b-b7d2-d4b6253f2e27/volumes" Jul 23 14:21:52.405344 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405312 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq"] Jul 23 14:21:52.405743 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405640 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8aa8acd-ca64-4f25-949c-01ea93e496e9" containerName="maas-api" Jul 23 14:21:52.405743 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405651 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8aa8acd-ca64-4f25-949c-01ea93e496e9" containerName="maas-api" Jul 23 14:21:52.405743 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405659 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef95338e-ff50-495b-b7d2-d4b6253f2e27" containerName="revoke-keys" Jul 23 14:21:52.405743 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405665 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef95338e-ff50-495b-b7d2-d4b6253f2e27" containerName="revoke-keys" Jul 23 14:21:52.405743 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405716 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8aa8acd-ca64-4f25-949c-01ea93e496e9" containerName="maas-api" Jul 23 14:21:52.405743 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.405726 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef95338e-ff50-495b-b7d2-d4b6253f2e27" containerName="revoke-keys" Jul 23 14:21:52.409941 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.409922 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.413142 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.413117 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-42e993-serving-cert\"" Jul 23 14:21:52.414426 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.414409 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 14:21:52.414705 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.414691 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-c7g9j\"" Jul 23 14:21:52.416891 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.416863 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 14:21:52.420001 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.419967 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq"] Jul 23 14:21:52.511136 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.511101 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.511306 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.511187 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.511306 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.511211 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56dfw\" (UniqueName: \"kubernetes.io/projected/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-kube-api-access-56dfw\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.612534 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.612504 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.612694 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.612577 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.612694 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.612599 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56dfw\" (UniqueName: \"kubernetes.io/projected/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-kube-api-access-56dfw\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.612769 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:21:52.612717 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-serving-cert: secret "maas-api-e2e-shared-a-42e993-serving-cert" not found Jul 23 14:21:52.612817 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:21:52.612790 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls podName:7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0 nodeName:}" failed. No retries permitted until 2026-07-23 14:21:53.112769955 +0000 UTC m=+2465.134481816 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls") pod "maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" (UID: "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0") : secret "maas-api-e2e-shared-a-42e993-serving-cert" not found Jul 23 14:21:52.613153 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.613136 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-openshift-service-ca\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:52.621877 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:52.621834 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56dfw\" (UniqueName: \"kubernetes.io/projected/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-kube-api-access-56dfw\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:53.117328 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:53.117297 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:53.117512 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:21:53.117442 2582 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-serving-cert: secret "maas-api-e2e-shared-a-42e993-serving-cert" not found Jul 23 14:21:53.117512 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:21:53.117508 2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls podName:7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0 nodeName:}" failed. No retries permitted until 2026-07-23 14:21:54.117491353 +0000 UTC m=+2466.139203210 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls") pod "maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" (UID: "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0") : secret "maas-api-e2e-shared-a-42e993-serving-cert" not found Jul 23 14:21:54.125942 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:54.125901 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:54.128347 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:54.128318 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") pod \"maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:54.220456 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:54.220424 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:54.353142 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:54.353120 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq"] Jul 23 14:21:54.355570 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:21:54.355543 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ee38110_d36f_4a6b_aa6e_8fc4e157c8d0.slice/crio-feb63e74085672275463d2791b92acc887e1dc4d8fdcd86da6c863a0ea9bb6ae WatchSource:0}: Error finding container feb63e74085672275463d2791b92acc887e1dc4d8fdcd86da6c863a0ea9bb6ae: Status 404 returned error can't find the container with id feb63e74085672275463d2791b92acc887e1dc4d8fdcd86da6c863a0ea9bb6ae Jul 23 14:21:54.477087 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:54.477007 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" event={"ID":"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0","Type":"ContainerStarted","Data":"feb63e74085672275463d2791b92acc887e1dc4d8fdcd86da6c863a0ea9bb6ae"} Jul 23 14:21:55.482753 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:55.482672 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" event={"ID":"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0","Type":"ContainerStarted","Data":"c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163"} Jul 23 14:21:55.483129 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:55.482780 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:21:55.504170 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:21:55.504118 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" podStartSLOduration=2.709380132 podStartE2EDuration="3.504097714s" totalImagesPullingTime="794.717582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:21:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:21:55.503380197 +0000 UTC m=+2467.525092078" watchObservedRunningTime="2026-07-23 14:21:55.504097714 +0000 UTC m=+2467.525809586" Jul 23 14:22:00.492354 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.492325 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr"] Jul 23 14:22:00.495886 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.495869 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.498477 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.498455 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-52507a-serving-cert\"" Jul 23 14:22:00.507726 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.507704 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr"] Jul 23 14:22:00.585290 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.585246 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvjgb\" (UniqueName: \"kubernetes.io/projected/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-kube-api-access-nvjgb\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.585458 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.585375 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.585458 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.585447 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.686138 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.686100 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.686348 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.686159 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.686348 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.686194 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvjgb\" (UniqueName: \"kubernetes.io/projected/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-kube-api-access-nvjgb\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.686867 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.686817 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-openshift-service-ca\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.688643 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.688620 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-maas-api-tls\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.695751 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.695726 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvjgb\" (UniqueName: \"kubernetes.io/projected/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-kube-api-access-nvjgb\") pod \"maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.805637 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.805551 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:00.934975 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:00.934946 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr"] Jul 23 14:22:00.937719 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:22:00.937679 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d7c79c1_3c33_4ee0_af60_8fac5c7580cb.slice/crio-aad0bad0575b394bd73970151572b18dd3b1a2fc45585d6e23455dcd51a4c52f WatchSource:0}: Error finding container aad0bad0575b394bd73970151572b18dd3b1a2fc45585d6e23455dcd51a4c52f: Status 404 returned error can't find the container with id aad0bad0575b394bd73970151572b18dd3b1a2fc45585d6e23455dcd51a4c52f Jul 23 14:22:01.493901 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:01.493869 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:22:01.515425 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:01.515389 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" event={"ID":"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb","Type":"ContainerStarted","Data":"7f45dd4e3573fd65c3430635b3d144d37e146071c5faf8b7cf5090e690a07b0f"} Jul 23 14:22:01.515600 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:01.515440 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" event={"ID":"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb","Type":"ContainerStarted","Data":"aad0bad0575b394bd73970151572b18dd3b1a2fc45585d6e23455dcd51a4c52f"} Jul 23 14:22:01.515600 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:01.515477 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:22:01.540226 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:01.540170 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" podStartSLOduration=1.080338388 podStartE2EDuration="1.540150329s" totalImagesPullingTime="459.811941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:22:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:22:01.538935244 +0000 UTC m=+2473.560647142" watchObservedRunningTime="2026-07-23 14:22:01.540150329 +0000 UTC m=+2473.561862210" Jul 23 14:22:07.525593 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:22:07.525562 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:25:48.583824 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:25:48.583793 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:25:48.592955 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:25:48.592934 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:25:48.601135 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:25:48.601088 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:25:48.605392 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:25:48.605373 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7fwh6_9a30c045-fceb-4feb-88e0-7e3c610061a8/ovn-acl-logging/0.log" Jul 23 14:25:48.693127 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:25:48.693099 2582 scope.go:117] "RemoveContainer" containerID="4bd51b3c98de722e974b34ec6f17497ef11bac74f44c62cd9ca0f83a12876809" Jul 23 14:26:48.713337 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:26:48.713302 2582 scope.go:117] "RemoveContainer" containerID="375a75025abf43b689a5153d5dbc6926268f736b7c56630405ade9cde113ab2b" Jul 23 14:27:48.734116 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:27:48.734080 2582 scope.go:117] "RemoveContainer" containerID="5f0d78b5113bb4cc4112cadf28b8f73c302a680b6ff86a24cf11a9c5c9a63643" Jul 23 14:27:48.742508 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:27:48.742487 2582 scope.go:117] "RemoveContainer" containerID="1c903bc8f1439682d18824db80906e7228b420725abb077c22a9e7aa72e11e01" Jul 23 14:28:04.116216 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.116180 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs"] Jul 23 14:28:04.119490 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.119473 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:04.122579 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.122560 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pngt\"" Jul 23 14:28:04.129998 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.129974 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs"] Jul 23 14:28:04.179591 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.179556 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5869427-cbe6-40cb-a162-eaa1a2278183-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs\" (UID: \"d5869427-cbe6-40cb-a162-eaa1a2278183\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:04.280289 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.280245 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5869427-cbe6-40cb-a162-eaa1a2278183-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs\" (UID: \"d5869427-cbe6-40cb-a162-eaa1a2278183\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:04.280977 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.280955 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5869427-cbe6-40cb-a162-eaa1a2278183-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs\" (UID: \"d5869427-cbe6-40cb-a162-eaa1a2278183\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:04.430355 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.430270 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:04.582059 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:04.582034 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs"] Jul 23 14:28:04.584569 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:28:04.584539 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5869427_cbe6_40cb_a162_eaa1a2278183.slice/crio-cdebd6fa353aba6ab000bed52ac91a617e9fbbd261e79cb1d1ed79dd68abc8cf WatchSource:0}: Error finding container cdebd6fa353aba6ab000bed52ac91a617e9fbbd261e79cb1d1ed79dd68abc8cf: Status 404 returned error can't find the container with id cdebd6fa353aba6ab000bed52ac91a617e9fbbd261e79cb1d1ed79dd68abc8cf Jul 23 14:28:05.206009 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:05.205968 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" event={"ID":"d5869427-cbe6-40cb-a162-eaa1a2278183","Type":"ContainerStarted","Data":"8f5edf1bd525463c92ca9eddd5ea53e164bc7273f94298912d072b681ddfff54"} Jul 23 14:28:05.206415 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:05.206036 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" event={"ID":"d5869427-cbe6-40cb-a162-eaa1a2278183","Type":"ContainerStarted","Data":"cdebd6fa353aba6ab000bed52ac91a617e9fbbd261e79cb1d1ed79dd68abc8cf"} Jul 23 14:28:05.225312 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:05.225265 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" podStartSLOduration=1.225249416 podStartE2EDuration="1.225249416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 14:28:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:28:05.224641687 +0000 UTC m=+2837.246353566" watchObservedRunningTime="2026-07-23 14:28:05.225249416 +0000 UTC m=+2837.246961295" Jul 23 14:28:06.212066 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:06.212032 2582 generic.go:356] "Generic (PLEG): container finished" podID="d5869427-cbe6-40cb-a162-eaa1a2278183" containerID="8f5edf1bd525463c92ca9eddd5ea53e164bc7273f94298912d072b681ddfff54" exitCode=0 Jul 23 14:28:06.212457 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:06.212093 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" event={"ID":"d5869427-cbe6-40cb-a162-eaa1a2278183","Type":"ContainerDied","Data":"8f5edf1bd525463c92ca9eddd5ea53e164bc7273f94298912d072b681ddfff54"} Jul 23 14:28:07.344860 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:07.344810 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:07.408153 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:07.408119 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5869427-cbe6-40cb-a162-eaa1a2278183-maas-api-service-ca\") pod \"d5869427-cbe6-40cb-a162-eaa1a2278183\" (UID: \"d5869427-cbe6-40cb-a162-eaa1a2278183\") " Jul 23 14:28:07.408454 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:07.408432 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5869427-cbe6-40cb-a162-eaa1a2278183-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d5869427-cbe6-40cb-a162-eaa1a2278183" (UID: "d5869427-cbe6-40cb-a162-eaa1a2278183"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:28:07.509265 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:07.509179 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5869427-cbe6-40cb-a162-eaa1a2278183-maas-api-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:08.222336 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:08.222308 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" Jul 23 14:28:08.222533 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:08.222302 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs" event={"ID":"d5869427-cbe6-40cb-a162-eaa1a2278183","Type":"ContainerDied","Data":"cdebd6fa353aba6ab000bed52ac91a617e9fbbd261e79cb1d1ed79dd68abc8cf"} Jul 23 14:28:08.222533 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:08.222471 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdebd6fa353aba6ab000bed52ac91a617e9fbbd261e79cb1d1ed79dd68abc8cf" Jul 23 14:28:14.554744 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.554709 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq"] Jul 23 14:28:14.555285 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.554980 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" podUID="7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" containerName="maas-api" containerID="cri-o://c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163" gracePeriod=30 Jul 23 14:28:14.805892 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.805804 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:28:14.876035 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.876006 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") pod \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " Jul 23 14:28:14.876240 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.876105 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56dfw\" (UniqueName: \"kubernetes.io/projected/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-kube-api-access-56dfw\") pod \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " Jul 23 14:28:14.876240 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.876139 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-openshift-service-ca\") pod \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\" (UID: \"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0\") " Jul 23 14:28:14.876551 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.876523 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" (UID: "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:28:14.878265 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.878229 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" (UID: "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:28:14.878265 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.878241 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-kube-api-access-56dfw" (OuterVolumeSpecName: "kube-api-access-56dfw") pod "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" (UID: "7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0"). InnerVolumeSpecName "kube-api-access-56dfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:28:14.977587 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.977552 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56dfw\" (UniqueName: \"kubernetes.io/projected/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-kube-api-access-56dfw\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:14.977587 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.977579 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:14.977587 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:14.977589 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:15.254516 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.254481 2582 generic.go:356] "Generic (PLEG): container finished" podID="7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" containerID="c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163" exitCode=0 Jul 23 14:28:15.254672 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.254529 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" Jul 23 14:28:15.254672 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.254571 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" event={"ID":"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0","Type":"ContainerDied","Data":"c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163"} Jul 23 14:28:15.254672 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.254628 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq" event={"ID":"7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0","Type":"ContainerDied","Data":"feb63e74085672275463d2791b92acc887e1dc4d8fdcd86da6c863a0ea9bb6ae"} Jul 23 14:28:15.254672 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.254653 2582 scope.go:117] "RemoveContainer" containerID="c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163" Jul 23 14:28:15.263608 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.263591 2582 scope.go:117] "RemoveContainer" containerID="c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163" Jul 23 14:28:15.263878 ip-10-0-128-225 kubenswrapper[2582]: E0723 14:28:15.263835 2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163\": container with ID starting with c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163 not found: ID does not exist" containerID="c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163" Jul 23 14:28:15.263950 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.263888 2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163"} err="failed to get container status \"c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163\": rpc error: code = NotFound desc = could not find container \"c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163\": container with ID starting with c337d3f0b269f677f13b7b68fb9eb4b1533a692824b305057fd82b6dfe223163 not found: ID does not exist" Jul 23 14:28:15.277922 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.277894 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq"] Jul 23 14:28:15.282071 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:15.282049 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-42e993-589d7df6dd-8ttjq"] Jul 23 14:28:16.563155 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:16.563119 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" path="/var/lib/kubelet/pods/7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0/volumes" Jul 23 14:28:19.203491 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:19.203451 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs"] Jul 23 14:28:19.205839 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:19.205815 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-42e993-6m6bs"] Jul 23 14:28:20.563048 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:20.563007 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5869427-cbe6-40cb-a162-eaa1a2278183" path="/var/lib/kubelet/pods/d5869427-cbe6-40cb-a162-eaa1a2278183/volumes" Jul 23 14:28:26.592596 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.592563 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv"] Jul 23 14:28:26.593026 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.592911 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" containerName="maas-api" Jul 23 14:28:26.593026 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.592922 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" containerName="maas-api" Jul 23 14:28:26.593026 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.592935 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5869427-cbe6-40cb-a162-eaa1a2278183" containerName="revoke-keys" Jul 23 14:28:26.593026 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.592941 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5869427-cbe6-40cb-a162-eaa1a2278183" containerName="revoke-keys" Jul 23 14:28:26.593026 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.593001 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5869427-cbe6-40cb-a162-eaa1a2278183" containerName="revoke-keys" Jul 23 14:28:26.593026 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.593011 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ee38110-d36f-4a6b-aa6e-8fc4e157c8d0" containerName="maas-api" Jul 23 14:28:26.597219 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.597201 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:26.599967 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.599949 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-6pngt\"" Jul 23 14:28:26.605004 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.604980 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv"] Jul 23 14:28:26.682801 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.682761 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv\" (UID: \"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:26.783633 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.783593 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv\" (UID: \"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:26.784274 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.784253 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv\" (UID: \"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:26.907633 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:26.907600 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:27.053394 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:27.053209 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv"] Jul 23 14:28:27.056237 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:28:27.056204 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea5e737a_2b1c_4f1a_bd8f_37144b8b8a31.slice/crio-2e9b6f6a3157b16a1bbaf4a95b72342c4c16a1e8f98339bc04b17153d5d4bbc1 WatchSource:0}: Error finding container 2e9b6f6a3157b16a1bbaf4a95b72342c4c16a1e8f98339bc04b17153d5d4bbc1: Status 404 returned error can't find the container with id 2e9b6f6a3157b16a1bbaf4a95b72342c4c16a1e8f98339bc04b17153d5d4bbc1 Jul 23 14:28:27.311386 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:27.311355 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" event={"ID":"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31","Type":"ContainerStarted","Data":"2e9b6f6a3157b16a1bbaf4a95b72342c4c16a1e8f98339bc04b17153d5d4bbc1"} Jul 23 14:28:28.317590 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:28.317501 2582 generic.go:356] "Generic (PLEG): container finished" podID="ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31" containerID="4fa9952f82f32f80fdf081e354563323e5bc63a316d2a16599fc4783188e0dcf" exitCode=0 Jul 23 14:28:28.317979 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:28.317593 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" event={"ID":"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31","Type":"ContainerDied","Data":"4fa9952f82f32f80fdf081e354563323e5bc63a316d2a16599fc4783188e0dcf"} Jul 23 14:28:29.453987 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:29.453962 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:29.612656 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:29.612628 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31-maas-api-service-ca\") pod \"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31\" (UID: \"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31\") " Jul 23 14:28:29.613041 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:29.613015 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31" (UID: "ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:28:29.714110 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:29.714075 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31-maas-api-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:30.328661 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:30.328636 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" Jul 23 14:28:30.328876 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:30.328676 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv" event={"ID":"ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31","Type":"ContainerDied","Data":"2e9b6f6a3157b16a1bbaf4a95b72342c4c16a1e8f98339bc04b17153d5d4bbc1"} Jul 23 14:28:30.328876 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:30.328729 2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e9b6f6a3157b16a1bbaf4a95b72342c4c16a1e8f98339bc04b17153d5d4bbc1" Jul 23 14:28:37.148541 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.148503 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr"] Jul 23 14:28:37.149204 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.148729 2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" podUID="6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" containerName="maas-api" containerID="cri-o://7f45dd4e3573fd65c3430635b3d144d37e146071c5faf8b7cf5090e690a07b0f" gracePeriod=30 Jul 23 14:28:37.364451 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.364421 2582 generic.go:356] "Generic (PLEG): container finished" podID="6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" containerID="7f45dd4e3573fd65c3430635b3d144d37e146071c5faf8b7cf5090e690a07b0f" exitCode=0 Jul 23 14:28:37.364650 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.364498 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" event={"ID":"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb","Type":"ContainerDied","Data":"7f45dd4e3573fd65c3430635b3d144d37e146071c5faf8b7cf5090e690a07b0f"} Jul 23 14:28:37.393424 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.393398 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:28:37.485273 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.485190 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nvjgb\" (UniqueName: \"kubernetes.io/projected/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-kube-api-access-nvjgb\") pod \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " Jul 23 14:28:37.485273 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.485228 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-openshift-service-ca\") pod \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " Jul 23 14:28:37.485510 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.485283 2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-maas-api-tls\") pod \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\" (UID: \"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb\") " Jul 23 14:28:37.485620 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.485586 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" (UID: "6d7c79c1-3c33-4ee0-af60-8fac5c7580cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 14:28:37.487282 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.487262 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" (UID: "6d7c79c1-3c33-4ee0-af60-8fac5c7580cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 14:28:37.487400 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.487379 2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-kube-api-access-nvjgb" (OuterVolumeSpecName: "kube-api-access-nvjgb") pod "6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" (UID: "6d7c79c1-3c33-4ee0-af60-8fac5c7580cb"). InnerVolumeSpecName "kube-api-access-nvjgb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 14:28:37.586225 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.586187 2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-maas-api-tls\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:37.586225 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.586220 2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nvjgb\" (UniqueName: \"kubernetes.io/projected/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-kube-api-access-nvjgb\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:37.586225 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:37.586230 2582 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb-openshift-service-ca\") on node \"ip-10-0-128-225.ec2.internal\" DevicePath \"\"" Jul 23 14:28:38.370571 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:38.370542 2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" Jul 23 14:28:38.370571 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:38.370550 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr" event={"ID":"6d7c79c1-3c33-4ee0-af60-8fac5c7580cb","Type":"ContainerDied","Data":"aad0bad0575b394bd73970151572b18dd3b1a2fc45585d6e23455dcd51a4c52f"} Jul 23 14:28:38.371118 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:38.370603 2582 scope.go:117] "RemoveContainer" containerID="7f45dd4e3573fd65c3430635b3d144d37e146071c5faf8b7cf5090e690a07b0f" Jul 23 14:28:38.399079 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:38.399049 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr"] Jul 23 14:28:38.406474 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:38.406451 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-52507a-5fcb59c84f-465sr"] Jul 23 14:28:38.564135 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:38.564104 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" path="/var/lib/kubelet/pods/6d7c79c1-3c33-4ee0-af60-8fac5c7580cb/volumes" Jul 23 14:28:41.709141 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:41.709106 2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv"] Jul 23 14:28:41.712243 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:41.712221 2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-52507a-bz2kv"] Jul 23 14:28:42.563088 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:42.563058 2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31" path="/var/lib/kubelet/pods/ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31/volumes" Jul 23 14:28:57.237654 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:57.237627 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c8b88fb69-xtfvz_65f63782-f830-4bd1-9131-90523f098744/copy-manifests/0.log" Jul 23 14:28:57.253246 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:57.253206 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c8b88fb69-xtfvz_65f63782-f830-4bd1-9131-90523f098744/manager/0.log" Jul 23 14:28:59.820961 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:28:59.820931 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w92bz_a68c23de-96ad-40a8-9e3f-5b2b18cde6f4/manager/0.log" Jul 23 14:29:00.251128 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:00.251097 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f4rrs_c8923431-61e9-44c6-8458-e33f6f0cdf6c/manager/0.log" Jul 23 14:29:01.071202 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:01.071170 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qp5bw_2e7a534b-7337-4f71-8f7d-b99e0ab7704f/discovery/0.log" Jul 23 14:29:01.759796 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:01.759758 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wx2xx_ad450e58-ddf8-4e57-bfc2-25c950b68509/payload-processing/0.log" Jul 23 14:29:02.560963 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:02.560896 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-dklfz_b8fd398e-3f9c-43b3-8c86-313ae302b4bc/main/0.log" Jul 23 14:29:02.816245 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:02.816137 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-w6bzc_95dca6a3-790e-45e8-a36e-ff4eada8b972/main/0.log" Jul 23 14:29:12.972200 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:12.972150 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rv5cp_2618d6b8-b672-4af6-bc7d-5e13639fc6a3/global-pull-secret-syncer/0.log" Jul 23 14:29:13.228136 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:13.228052 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fr56v_fdc090cc-932e-4c22-b4cc-c3cc789fbce0/konnectivity-agent/0.log" Jul 23 14:29:13.402965 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:13.402937 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-225.ec2.internal_a1cab1d6084d8841ec0ac45f2a8bd856/haproxy/0.log" Jul 23 14:29:18.592463 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:18.592425 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w92bz_a68c23de-96ad-40a8-9e3f-5b2b18cde6f4/manager/0.log" Jul 23 14:29:18.897063 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:18.897028 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f4rrs_c8923431-61e9-44c6-8458-e33f6f0cdf6c/manager/0.log" Jul 23 14:29:21.156619 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.156588 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64d8476f76-z2ztf_389aaeb0-01ed-4b02-a94e-6c3b8992cc00/metrics-server/0.log" Jul 23 14:29:21.447156 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.447067 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wwjhj_800e28ee-361c-4a25-9120-9d33513a56d2/node-exporter/0.log" Jul 23 14:29:21.471551 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.471525 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wwjhj_800e28ee-361c-4a25-9120-9d33513a56d2/kube-rbac-proxy/0.log" Jul 23 14:29:21.496877 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.496837 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wwjhj_800e28ee-361c-4a25-9120-9d33513a56d2/init-textfile/0.log" Jul 23 14:29:21.617766 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.617729 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/0.log" Jul 23 14:29:21.618986 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.618965 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/prometheus/1.log" Jul 23 14:29:21.645638 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.645609 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/config-reloader/0.log" Jul 23 14:29:21.671392 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.671362 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/thanos-sidecar/0.log" Jul 23 14:29:21.703962 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.703891 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/kube-rbac-proxy-web/0.log" Jul 23 14:29:21.735436 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.735412 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/kube-rbac-proxy/0.log" Jul 23 14:29:21.767704 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.767678 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/kube-rbac-proxy-thanos/0.log" Jul 23 14:29:21.817615 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.817595 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_fa81ddc9-1b21-4540-b18a-5a0fdaf16d6f/init-config-reloader/0.log" Jul 23 14:29:21.893530 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.893505 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-vvvkw_c59cf886-f0fb-4335-9254-46dcbf24ba51/prometheus-operator/0.log" Jul 23 14:29:21.949482 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:21.949456 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-vvvkw_c59cf886-f0fb-4335-9254-46dcbf24ba51/kube-rbac-proxy/0.log" Jul 23 14:29:22.116050 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.116012 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747ddb4dbd-zt5qn_96abc0a4-e39a-461c-9177-d0b845c4aed4/thanos-query/0.log" Jul 23 14:29:22.148852 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.148817 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747ddb4dbd-zt5qn_96abc0a4-e39a-461c-9177-d0b845c4aed4/kube-rbac-proxy-web/0.log" Jul 23 14:29:22.173289 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.173261 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747ddb4dbd-zt5qn_96abc0a4-e39a-461c-9177-d0b845c4aed4/kube-rbac-proxy/0.log" Jul 23 14:29:22.198529 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.198505 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747ddb4dbd-zt5qn_96abc0a4-e39a-461c-9177-d0b845c4aed4/prom-label-proxy/0.log" Jul 23 14:29:22.235941 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.235910 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747ddb4dbd-zt5qn_96abc0a4-e39a-461c-9177-d0b845c4aed4/kube-rbac-proxy-rules/0.log" Jul 23 14:29:22.275637 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.275613 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-747ddb4dbd-zt5qn_96abc0a4-e39a-461c-9177-d0b845c4aed4/kube-rbac-proxy-metrics/0.log" Jul 23 14:29:22.663280 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663247 2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz"] Jul 23 14:29:22.663586 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663575 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" containerName="maas-api" Jul 23 14:29:22.663631 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663588 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" containerName="maas-api" Jul 23 14:29:22.663631 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663601 2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31" containerName="revoke-keys" Jul 23 14:29:22.663631 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663607 2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31" containerName="revoke-keys" Jul 23 14:29:22.663724 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663658 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea5e737a-2b1c-4f1a-bd8f-37144b8b8a31" containerName="revoke-keys" Jul 23 14:29:22.663724 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.663667 2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d7c79c1-3c33-4ee0-af60-8fac5c7580cb" containerName="maas-api" Jul 23 14:29:22.666790 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.666773 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.670978 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.670959 2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jrxnq\"/\"default-dockercfg-7h9v2\"" Jul 23 14:29:22.672080 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.672062 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jrxnq\"/\"kube-root-ca.crt\"" Jul 23 14:29:22.672769 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.672410 2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jrxnq\"/\"openshift-service-ca.crt\"" Jul 23 14:29:22.682958 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.682933 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz"] Jul 23 14:29:22.766368 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.766338 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-lib-modules\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.766368 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.766374 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-sys\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.766571 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.766402 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-proc\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.766571 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.766420 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4prbh\" (UniqueName: \"kubernetes.io/projected/1c58a7a5-2ebf-44f4-a106-e005e623d930-kube-api-access-4prbh\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.766571 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.766444 2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-podres\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867616 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867579 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-proc\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867616 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867616 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4prbh\" (UniqueName: \"kubernetes.io/projected/1c58a7a5-2ebf-44f4-a106-e005e623d930-kube-api-access-4prbh\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867863 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867653 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-podres\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867863 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867706 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-proc\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867863 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867727 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-lib-modules\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867863 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867757 2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-sys\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.867863 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867834 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-sys\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.868073 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867832 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-podres\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.868073 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.867943 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c58a7a5-2ebf-44f4-a106-e005e623d930-lib-modules\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.878235 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.878204 2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4prbh\" (UniqueName: \"kubernetes.io/projected/1c58a7a5-2ebf-44f4-a106-e005e623d930-kube-api-access-4prbh\") pod \"perf-node-gather-daemonset-lzggz\" (UID: \"1c58a7a5-2ebf-44f4-a106-e005e623d930\") " pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:22.976070 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:22.975975 2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:23.111547 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:23.111524 2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz"] Jul 23 14:29:23.113361 ip-10-0-128-225 kubenswrapper[2582]: W0723 14:29:23.113331 2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod1c58a7a5_2ebf_44f4_a106_e005e623d930.slice/crio-102361ecee851b5b9383a4f2f5e58e2ece757a9b4a0dc53d60b2cfc1e4fe2145 WatchSource:0}: Error finding container 102361ecee851b5b9383a4f2f5e58e2ece757a9b4a0dc53d60b2cfc1e4fe2145: Status 404 returned error can't find the container with id 102361ecee851b5b9383a4f2f5e58e2ece757a9b4a0dc53d60b2cfc1e4fe2145 Jul 23 14:29:23.577975 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:23.577885 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" event={"ID":"1c58a7a5-2ebf-44f4-a106-e005e623d930","Type":"ContainerStarted","Data":"2741a07a92debd098a360bb5c8bc1116d096276a60d0a5c1547f87edf0a55c54"} Jul 23 14:29:23.577975 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:23.577942 2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" event={"ID":"1c58a7a5-2ebf-44f4-a106-e005e623d930","Type":"ContainerStarted","Data":"102361ecee851b5b9383a4f2f5e58e2ece757a9b4a0dc53d60b2cfc1e4fe2145"} Jul 23 14:29:23.578405 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:23.578077 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:23.613517 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:23.613467 2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" podStartSLOduration=1.613451388 podStartE2EDuration="1.613451388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 14:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 14:29:23.612301499 +0000 UTC m=+2915.634013382" watchObservedRunningTime="2026-07-23 14:29:23.613451388 +0000 UTC m=+2915.635163266" Jul 23 14:29:26.708857 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:26.708816 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wndd6_9c6878de-2105-4646-9e71-5771b0fb9834/dns/0.log" Jul 23 14:29:26.757937 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:26.757907 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wndd6_9c6878de-2105-4646-9e71-5771b0fb9834/kube-rbac-proxy/0.log" Jul 23 14:29:26.823579 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:26.823550 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dmf95_0435abd0-6478-44ee-ba15-68e22040a043/dns-node-resolver/0.log" Jul 23 14:29:27.453190 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:27.453158 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6fb5c8cbbc-v957v_582be53d-6561-48f7-aebb-7e3be8c9a208/registry/0.log" Jul 23 14:29:27.530352 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:27.530321 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-s5m98_4bfa6ee4-b6e5-432c-8152-4c008391612b/node-ca/0.log" Jul 23 14:29:28.753821 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:28.753776 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qp5bw_2e7a534b-7337-4f71-8f7d-b99e0ab7704f/discovery/0.log" Jul 23 14:29:28.962737 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:28.962707 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wx2xx_ad450e58-ddf8-4e57-bfc2-25c950b68509/payload-processing/0.log" Jul 23 14:29:29.591698 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:29.591672 2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jrxnq/perf-node-gather-daemonset-lzggz" Jul 23 14:29:29.761771 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:29.761740 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pdx59_25122a4b-972d-4b3c-aca2-5d6bee9cb90b/serve-healthcheck-canary/0.log" Jul 23 14:29:30.267765 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:30.267738 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8ffrm_f571e772-75de-477b-8383-a46e1de71d1d/kube-rbac-proxy/0.log" Jul 23 14:29:30.287753 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:30.287730 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8ffrm_f571e772-75de-477b-8383-a46e1de71d1d/exporter/0.log" Jul 23 14:29:30.309011 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:30.308984 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8ffrm_f571e772-75de-477b-8383-a46e1de71d1d/extractor/0.log" Jul 23 14:29:33.318452 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:33.318371 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qp5bw_2e7a534b-7337-4f71-8f7d-b99e0ab7704f/discovery/0.log" Jul 23 14:29:33.507860 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:33.507813 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wx2xx_ad450e58-ddf8-4e57-bfc2-25c950b68509/payload-processing/0.log" Jul 23 14:29:33.921229 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:33.921204 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c8b88fb69-xtfvz_65f63782-f830-4bd1-9131-90523f098744/manager/0.log" Jul 23 14:29:33.961605 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:33.961583 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c8b88fb69-xtfvz_65f63782-f830-4bd1-9131-90523f098744/copy-manifests/0.log" Jul 23 14:29:35.036302 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:35.036274 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qp5bw_2e7a534b-7337-4f71-8f7d-b99e0ab7704f/discovery/0.log" Jul 23 14:29:35.264519 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:35.264490 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6d8d9c69b6-dfkr4_d5b7e999-798c-4a5c-bdf3-fe5fa64ba65b/manager/0.log" Jul 23 14:29:35.342012 ip-10-0-128-225 kubenswrapper[2582]: I0723 14:29:35.341940 2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-wx2xx_ad450e58-ddf8-4e57-bfc2-25c950b68509/payload-processing/0.log"